sitemap-ts

Sitemap generator for TypeScript projects

v1.9.0 URL: https://unpkg.com/sitemap-ts@1.9.0/dist/index.cjs

Open Browse Files

Sitemap Typescript Generator Robots Sitemap.xml Robots.txt

sitemap-ts-advanced

Sitemap generator for TypeScript projects

v0.2.0 URL: https://unpkg.com/sitemap-ts-advanced@0.2.0/dist/index.cjs

Open Browse Files

Sitemap Typescript Generator Robots Sitemap.xml Robots.txt

microfrontier

A web crawler frontier implementation in TypeScript backed by Redis. MicroFrontier is a scalable and distributed frontier implemented through Redis Queues.

v1.0.2 URL: https://unpkg.com/microfrontier@1.0.2/dist/index.js

Open Browse Files

Web Crawler Frontier Search Engine URL Redis Queue Spider Robots.txt

norobots

Robots.txt rules

v0.1.0 URL: https://unpkg.com/norobots@0.1.0/index.js

Open Browse Files

Robots.txt node.js Web crawler bot