# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file User-agent: * Disallow: /register Disallow: /register-adviser Disallow: /search Disallow: /manage Disallow: /upload Disallow: /consumer-inquiry Disallow: /article-type Disallow: /album Disallow: /profile Disallow: /feed Disallow: /chat Disallow: /messages Disallow: /links Disallow: /notice Disallow: /account Disallow: /client Disallow: /info Disallow: /usericon Disallow: /upgrade Sitemap: http://www.2100bx.com/map/sitemap_index.xml User-agent: Slurp User-agent: Sosospider Crawl-delay: 5 Request-rate: 12/1m 0100 - 1659 Request-rate: 40/1m 1700 - 0059