# A standard robots.txt file that controls # access to the site for crawlers User-agent: * Disallow: /aspnet_client/ Disallow: /_private/ Disallow: /_vti_cnf/ Disallow: /_vti_pvt/ Disallow: /files/