Morning,
I seem to having problems with my robots.txt file and Google Webmaster, i keep getting the following error;
'Some important page is blocked by robots.txt. More Details'
I've tried various combinations but it still comes back with - 200 (Success) Googlebot is blocked from http://www.domainname.co.uk/
Is there something i need to update within the htacces file?
Regards,
Michael
David,
You are right, it was a wordpress plugin which was over-writing the original robots.txt
Here's hoping Google is now happy... :)
Thanks,
Michael
Hi Michael,
There isn't a robots.txt included in the PriceTapestry.org distribution so this sounds like it is referring to your main robots.txt in the top level of your website; and furthermore that it contains
User-Agent: *Disallow: /
Check if the above is present, and if so remove those lines; or if not - check for Disallow: / on it's own and confirm that it is relevant only to those crawlers you do actually want to block. If you're not sure post the contents of your robots.txt and I'll check it out for you...
Cheers,
David.
--
PriceTapestry.com