# CRELoaded Generated Robots.txt # Robot Exclusion File -- robots.txt # Author: CRELoaded Team # Last Updated : September 11th, 2009 User-Agent: * Disallow: /themes/ Disallow: /config/ Disallow: /huadmin/ Disallow: /modules/ #To block main page due to size from bots #Disallow: /index.php #Block out things that are secure or login oriented #These are based on not using any rewrite rules Disallow: /address.php Disallow: /cart.php Disallow: /email.php Disallow: /order.php Disallow: /password.php Disallow: /order-detail.php # IF YOU DO NOT WISH TO HAVE THE GOOGLE IMAGE BOT SCAN YOUR DOMAIN FOR IMAGES # THEN YOU CAN INCLUDE THE FOLLOWING IN YOUR ROBOTS FILE. # I FOUND THAT MY BANDWIDTH USAGE DROPPED BY A MASSIVE AMOUNT AFTER I GOT RID # OF THE GOOGLE IMAGE BOT. ALL I HAD WAS IMAGE HUNTERS STEALING PRODUCT SHOTS # AND NOT EVEN BROWSING THE SITE. #User-agent: Googlebot-Image #Disallow: /