simplecrawler

Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

v1.1.9 URL: https://unpkg.com/simplecrawler@1.1.9/lib/index.js

Open Browse Files

simple crawler spider cache queue simplecrawler eventemitter

simplecrawler-zic

Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

v1.2.6 URL: https://unpkg.com/simplecrawler-zic@1.2.6/lib/index.js

Open Browse Files

simple crawler spider cache queue simplecrawler eventemitter

simplecrawler-mongo-queue

MongoDB queue implementation for simplecrawler

v1.0.1 URL: https://unpkg.com/simplecrawler-mongo-queue@1.0.1/index.js

Open Browse Files

simplecrawler mongo queue

@gombosg/simplecrawler

Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

v1.1.9 URL: https://unpkg.com/@gombosg/simplecrawler@1.1.9/lib/index.js

Open Browse Files

simple crawler spider cache queue simplecrawler eventemitter

sqlite-simplecrawler-queue

SQLite FetchQueue Implementation for Simplecrawler

v1.1.0 URL: https://unpkg.com/sqlite-simplecrawler-queue@1.1.0/startcrawler.js

Open Browse Files

sqlite simplecrawler queue