aragog-client

Aragog web scraping framework client

v1.0.3 URL: https://unpkg.com/aragog-client@1.0.3/lib/index.js

Open Browse Files

scraping crawling crawler spider distributed amqp framework

bulldozer-c

分布式爬虫

v0.0.72 URL: https://unpkg.com/bulldozer-c@0.0.72/index.js

Open Browse Files

爬虫分布式爬虫分布式 spider Distributed Distributed Spider scrapy

floodesh

Floodesh is a distributed web spider/crawler written with Nodejs.

v0.8.19 URL: https://unpkg.com/floodesh@0.8.19/index.js

Open Browse Files

nodejs distributed web spider crawler

spiderhunt

Very straightforward web crawler. Uses EventEmitter. Based on Simplecrawler but using a distributed queue system.

v0.1.2 URL: https://unpkg.com/spiderhunt@0.1.2/lib/index.js

Open Browse Files

simple crawler spider cache queue simplecrawler eventemitter spiderhunt distributed redis rabbitmq rabbit scalable

floodesh-cli

a cli tool for floodesh

v0.4.22 URL: https://unpkg.com/floodesh-cli@0.4.22/index.js

Open Browse Files

cli tool floodesh scaffold spider crawler distributed