# Don't expect to find any "secret" information here; # these pages would be worthless for a search engine to index. User-agent: Slurp Crawl-delay: 5 User-agent: msnbot Crawl-delay: 320 User-agent: * Disallow: /TEST/ Disallow: Crawl-delay: 300