robots-txt-component

Lightweight robots.txt parsing component without any external dependencies for Node.js.