Restricted by robots.txt does this cause problems?

[ocelot]

I have restricted around 1,500 links which are links to retailers website and links that affiliate links accorsing to webmaster tools

Is this the right approach as I thought it would affect the link juice? or should I take the no follow out of the restricted by robots.txt file

[RankSurge]

From a coding perspective, applying the nofollow to the links is the best way to go.

With the robots.txt file, only the top tier search engines respect the information contained within, so lesser known bots or spammers might check your robots.txt file to see what you don't want listed, and that info will give them a starting point to look deeper into your site.

[Everett]

Hello Ocelot,

I am assuming you have a site that has affiliate links and you want to keep Google from crawling those affiliate links. If I am wrong, please let me know. Going forward with that assumption then...

That is one way to do it. So perhaps you first send all of those links through a redirect via a folder called /out/ or /links/ or whatever, and you have blocked that folder in the robots.txt file. Correct? If so, this is how many affiliate sites handle the situation.

I would not rely on rel nofollow alone, though I would use that in addition to the robots.txt block.

There are many other ways to handle this. For instance, you could make all affilaite links javascript links instead of href links. Then you could put the javascript into a folder called /js/ or something like that, and block that in the robots.txt file. This works less and less now that Google Preview Bot seems to be ignoring the disallow statement in those situations.

You could make it all the same URL with a unique identifyer of some sort that tells your database where to redirect the click. For example:

www.yoursite.com/outlink/mylink#123

or

www.yoursite.com/mylink?link-id=123

In which case you could then block /mylink in the robots.txt file and tell Google to ignore the link-ID parameter via Webmaster Tools.

As you can see, there is more than one way to skin this cat. The problem is always going to be doing it without looking like you're trying to "fool" Google - because they WILL catch up with any tactic like that eventually.

Good luck!

Everett