# This is a file retrieved by spiders that conform to a defacto standard # to tell them what parts of the site they're allowed to crawl. The format # is # # User-agent: # Disallow: | # # (see ). # User-agent: * Disallow: /members/ # This robot seems overly aggressive, so out it goes #User-Agent: sitecheck.internetseer.com #Disallow: /