# Robots.txt file from http://www.chem99.com # All robots will spider the domain User-agent: * Disallow: /channel2 Disallow: /price2 Disallow: /*PageStart Disallow: /*default.aspx Disallow: /*?l=1$ Disallow: /Default.aspx Disallow: /*Default.aspx Disallow: /*?bdclkid