robots

Parser for robots.txt

Browse on unpkg