Download heritrix hdfs writer Free Java Code

Description

A Heritrix 3 writer processor for storing crawled data in the Hadoop Distributed File System.

Source Files

The download file heritrix-hdfs-writer-master.zip has the following entries.


.classpath/*from ww  w  .  j  av a 2  s .c om*/
.gitignore
.project
CHANGELOG.txt
README.txt
build.xml
lib/archive-overlay-commons-httpclient-3.1.jar
lib/archive-overlay-commons-pool-1.3.jar
lib/commons-httpclient-3.0.1.jar
lib/commons-lang-2.3.jar
lib/fastutil-5.0.7.jar
lib/hadoop-0.20.1-core.jar
lib/heritrix-commons-3.0.0.jar
lib/heritrix-engine-3.0.0.jar
lib/heritrix-modules-3.0.0.jar
lib/json-20090211.jar
lib/log4j-1.2.15.jar
lib/spring-beans-2.5.6.jar
lib/spring-context-2.5.6.jar
lib/spring-core-2.5.6.jar
src/.classpath
src/.project
src/conf/modules/Processor.options
src/java/.classpath
src/java/.project
src/java/org/archive/io/hdfs/HDFSParameters.java
src/java/org/archive/io/hdfs/HDFSWriter.java
src/java/org/archive/io/hdfs/HDFSWriterDocument.java
src/java/org/archive/io/hdfs/HDFSWriterFactory.java
src/java/org/archive/io/hdfs/HDFSWriterPool.java
src/java/org/archive/io/hdfs/HDFSWriterPoolMember.java
src/java/org/archive/io/hdfs/util/ByteScan.java
src/java/org/archive/io/hdfs/util/ByteSeq.java
src/java/org/archive/io/hdfs/util/HTML.java
src/java/org/archive/io/hdfs/util/URIUtils.java
src/java/org/archive/modules/writer/HDFSWriterProcessor.java