robotstxt

★ 0 updated 6y ago ⑂ fork

The robots.txt exclusion protocol implementation for Go language

No plain-English explanation yet — one is being written right now. Check back in a minute.