# $Id$ # robots.txt RFC: http://www.robotstxt.org/wc/norobots-rfc.html # Validated by http://www.searchengineworld.com/cgi-bin/robotcheck.cgi User-agent: Mediapartners-Google* User-agent: * Crawl-delay: 200 User-agent: ia_archiver Disallow: