##User-agent: Slurp
## Do we want to ban or limit other sites even more?
## http://help.yahoo.com/l/us/yahoo/search/webcrawler/slurp-03.html

User-Agent: *
Crawl-delay: 10
Disallow: /cgi-bin/
Disallow: /ftp/
Disallow: /bugzilla/
Request-rate: 1/10         # maximum rate is one page every 10 seconds
Visit-time: 0001-0600     # only visit between 00:01 and 06:00 UTC (GMT)