Adding the link masking directory to robots.txt?

[irdeto]

Hey guys,

Just want to know if you have any experience with this.

Is it worthwhile blocking search engines from following the link masking directory.. (what i mean by this is the directory that holds the link redirectors to an affiliate site:

example:
mydomain.com/go/thislink

goes to 
amazon.com/affiliatelink

I want to know if blocking the 'go' directory from getting crawled in robots.txt is a good idea or a bad idea?

I am not using wordpress but rather a custom built php site where i need to manually decide on these things.

i want to specifically know if this in any way violates guidelines for google. it doesn't change the custom experience because they know exactly where they will end up if they click on the link.

any advice would be much appreciated.

[wanatop]

Hi!

I know cases where you can hide the contents of a folder to Google to not affect the rest of the website.

Note that you are facing a machine that knows how to crawl and index content.

Also be sure to block with noindex nofollow meta tags and removing URLs from Google index if at some point you had not locked.

Best regards!

[Everett]

Iredeto,

I do this on a few of my sites and it works out well. Saves on crawl budget, keeps google from accessing my affiliate links, and keeps any pagerank from passing through the links, which keeps me in-line with Google's webmaster policy on links.

One thing to keep in mind is that Google may show those URLs in the search results with a message that they can't show the content because it has been blocked. If you want to keep this from happening you may have to remove that directory GWT using the URL Removal tool. If they get re-indexed in 90 days (or whatever the reset time frame is) you will have to do it again. Hopefully that won't be an issue once you get them all removed the first time and block the folder.

Using rel = "nofollow" tags on the hyperlinks going to that directory wouldn't hurt either.