Robots.txt

[PeterM22]

Hi there,

My question relates to the robots.txt file. This statement:

/*/trackback

Would this block domain.com/trackback and domain.com/fred/trackback ?

Peter

[RyanKent]

Peter,

I just completed some tests in order to answer your question.

Disallow: /*/trackback would block domain.com/fred/trackback but would NOT block domain.com/trackback

[PeterM22]

Great, thanks Ryan. I wondered if that would be the case, but wasn't sure. I'm curious how did you conduct that test?

[RyanKent]

I added a page to an existing site named trackback.php. I generated a sitemap using xml sitemaps and did not see the page. I added a link to /trackback from an existing page on the site then generated a new sitemap where I saw the page.

Finally, I added the entry into the robots.txt file and generated another sitemap. The page no longer appeared.

[Ontarioseo]

You can test the robots.txt file in Google Webmaster tools. It's under "Crawler access" in the "Site Configuration" section. Input the robots.txt content and the directories you want to test and it will give you feedback on the status.