# Robots.txt file allows all webcrawlers to index this site # # All robots will spider the domain. # The "Courses" directory is not spider'ed because there may be quizzes etc. in there # that instructors wish not to be made public. # We disallow the cgi-bin because it's just good practice not to allow the world to know what # executables are on the server. # Same goes for the PHPMyAdmin directory User-agent: * Disallow: /Courses/ Disallow: /cgi-bin/ Disallow: /phpMyAdmin/ Disallow: /CMU_Only/ Disallow: /MLRC/Reservation/ Disallow: /catalog/