Download roach Free Java Code
Description
A very adaptable web crawler framework. Impossible to kill.
Source Files
The download file roach-master.zip has the following entries.
.gitignore//from w w w . ja va 2s . c o m
.travis.yml
Gruntfile.js
LICENSE
README.md
docs/img/crawler_architecture.jpg
docs/img/crawler_data_flow.jpg
index.js
lib/filters/directory.js
lib/filters/links.js
lib/filters/trim.js
lib/filters/zip.js
lib/job.js
lib/parser.js
lib/proxy.js
lib/roach.js
lib/transports/file.js
lib/transports/logger.js
lib/transports/mongodb.js
lib/transports/rabbitmq.js
lib/transports/redis.js
package.json
roach_medium.png
test/filters/directory.test.js
test/filters/links.test.js
test/lib/parser.test.js
test/lib/proxy.test.js
test/utils/date.test.js
test/utils/request.test.js
test/utils/schedule.test.js
test/utils/string.test.js
utils/date.js
utils/index.js
utils/request.js
utils/schedule.js
utils/string.js
utils/timezones.js
Download
Click the following link to download roach-master.zip.
roach-master.zip