simplecrawler

Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

buffered-queue

A simple to use buffering queue (no dependencies) which flexibly buffers objects, strings, integers etc. until either a maximum size is reached, or an interval has come to an end.