Search

sitemapper

Parser for XML Sitemaps to be used with Robots.txt and web crawlers

v3.2.18 URL: https://unpkg.com/sitemapper@3.2.18/lib/assets/sitemapper.js

OpenBrowse Files

parsesitemapxmlrobots.txtsitemapscrawlerswebcrawler

@qbbsh/sitemapper

Parser for XML Sitemaps to be used with Robots.txt and web crawlers

v3.2.10 URL: https://unpkg.com/@qbbsh/sitemapper@3.2.10/lib/assets/sitemapper.js

OpenBrowse Files

parsesitemapxmlrobots.txtsitemapscrawlerswebcrawler

robots-parser

A specification compliant robots.txt parser with wildcard (*) matching support.

v3.0.1 URL: https://unpkg.com/robots-parser@3.0.1/index.js

OpenBrowse Files

robots.txtparseruser-agentscraperspiderbotrobots-exclusion-standard

html-snapshots

A selector-based html snapshot tool using Puppeteer or PhantomJS that sources sitemap.xml, robots.txt, or arbitrary input

v4.9.0 URL: https://unpkg.com/html-snapshots@4.9.0/lib/html-snapshots.js

OpenBrowse Files

SEOhtmlsnapshotsselectorajaxSPArobots.txtsitemap.xml

sitemap-ts

Sitemap generator for TypeScript projects

v1.8.0 URL: https://unpkg.com/sitemap-ts@1.8.0/dist/index.cjs

OpenBrowse Files

SitemapTypescriptGeneratorRobotsSitemap.xmlRobots.txt