# Default exclusions User-agent: * Disallow: /legal-notice/ Disallow: /extensions/ Disallow: /manifest/ Disallow: /symphony/ Disallow: /workspace/ Disallow: /*.avi$ Disallow: /*.css$ Disallow: /*.db$ Disallow: /*.exe$ Disallow: /*.htc$ Disallow: /*.inc$ Disallow: /*.ini$ Disallow: /*.js$ Disallow: /*.js.php$ Disallow: /*.log$ Disallow: /*.mp3$ Disallow: /*.mpg$ Disallow: /*.mpeg$ Disallow: /*.png$ Disallow: /*.psd$ Disallow: /*.rar$ Disallow: /*.wav$ Disallow: /*.zip$ #Despictable and evil robots to keep out :) User-agent: Slurp Crawl-delay: 30 User-agent: Teoma Crawl-delay: 60 User-Agent: MJ12bot Crawl-Delay: 120