User-agent: * Disallow: *mailto* Disallow: */feed/ Disallow: */admin/ # disable following crawlers User-agent: Steeler Disallow: / # reason: disregard this robot rule, so just block it now. #User-agent: MJ12bot #Disallow: / User-agent: IRLbot Disallow: / User-agent: yetibot Disallow: / User-agent: Googlebot Disallow: /wcs/ Disallow: /forum/ Disallow: /bookstore/ Disallow: /bookstore_us/ Disallow: /album/ Disallow: /cgi-bin/ Disallow: /Chinaneko/ Disallow: /chinaneko/ Disallow: /calendar Disallow: /diary/ Disallow: /fortune-telling/ Disallow: /misc/ Disallow: /media/