simplecrawler

Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

v1.1.9 URL: https://unpkg.com/simplecrawler@1.1.9/lib/index.js

Open Browse Files

simple crawler spider cache queue simplecrawler eventemitter

semantic-crawler

Priority based Semantic Web Crawler.

v0.0.2 URL: https://unpkg.com/semantic-crawler@0.0.2/semantic-crawler.js

Open Browse Files

semantic-crawler pdf-crawler text-crawler priority priority-crawler scraper crawling spider scraper scraping simplecrawler crawler osmosis js-crawler supercrawler node-web-crawler web-crawler

simplecrawler-zic

Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

v1.2.6 URL: https://unpkg.com/simplecrawler-zic@1.2.6/lib/index.js

Open Browse Files

simple crawler spider cache queue simplecrawler eventemitter

mongodb-simplecrawler-queue

MongoDB FetchQueue Implementation for Simplecrawler

v0.4.1 URL: https://unpkg.com/mongodb-simplecrawler-queue@0.4.1/lib/src/index.js

Open Browse Files

mongo simplecrawler queue

web-spider

web-spider is a simple and fast web spider written with Nodejs!

v0.3.1 URL: https://unpkg.com/web-spider@0.3.1/index.js

Open Browse Files

crawler spider phantomjs superagent jquery simplecrawler simplespider cheerio