Hi,
Can you advice me on the pt folders under a Wordpress multisite?
I want to avoid duplicate content penalties by Google and was thinking about two things to prevent searchengine finding my pt folders:
1. Added in robots.txt on root in public_html: Disallow: /pt_*/
2. Put an index.php in all /pt_*/ folders with:
<?php
header( "Status: 301 Moved Permanently" );
header( "Location: http://myWPmainsite.nl/" );
exit(0); //
?>
My questions are:
a. Is this not causing problems with the PTO plugin functionality? Seems okay, but maybe I overseen something?
b. Any more suggestions to prevent the pt folders being found/crawled?
Hi,
Shouldn't I use a wildcard?
For example if I have two pt installations?
mydomainname/pt_1/
mydomainname/pt_2/
Should it be Disallow: /pt/
or
Disallow: /pt
or
Disallow: /pt*/
?
Hi Marco,
robots.txt Disallow directives are simple "beginning of line" matches - so the wildcard is implied. This means that the trailing "/" must be included, otherwise
Disallow: /pt
...would inadvertantly also block anything beginning with /pt for example
www.example.com/ptfoo.html
...so the correct exclusion is:
Disallow: /pt/
Cheers,
David.
--
PriceTapestry.com
I'm not sure if I understand it correctly? If I have exclusion /pt/ it wouldn't exclude /pt_1/ and /pt_2/ ? So shouldn't the exclusion be /pt_ instead?
Sorry Marco, I was replying generically had overlooked your multisite Price Tapestry installation, in which case - exactly as you posted, a single top-level robots.txt exclusion using:
Disallow: /pt_Cheers,
David.
--
PriceTapestry.com
Hi Marco,
In your top level robots.txt, use:
User-Agent: *Disallow: /pt/
(or just add the Disallow: /pt/ line to any existing Disallow list for all user agents).
Then, in your /pt/ folder, you can DELETE all files except:
config.php
config.advanced.php
jump.php
I would personally do that, and just let anything that does request /pt/ directly just get 404 (Not Found). I'm not a big fan of 301 redirects as these have been known to have strange affects in search engines in the past!
Cheers,
David.
--
PriceTapestry.com