Blocking links from being crawled

[uesat]

For my blog, should I block category, "read more" and author links from being crawled by search engine robots?  Is it good for SEO?

Thanks

[ALLee]

Hi,

I wouldn't block these pages from being crawled by search engines. Category pages are great for making sure more link juice flows to your deeper blog pages and making sure they get indexed. I believe author links give authorrank to the corresponding blog post too. I'm not sure about what you mean by 'read more' links. May I ask why you are concerned that these pages hurt SEO?

[David_ODonnell]

If possible add unique content to your category pages, and use unique post extracts. Then no need to block.

Don't block "read more" as these link to full posts.

[MoosaHemani]

Category, you can block it, if you see they are creating duplicate content to the website.

Read More: Why block this? I mean it is link to your post and this is very much ok so I personally will not recommend you this!

Author Link: Hello No, Infect making it more interactive can allow you inspire new readership!

[uesat]

I'm concerned because my website lost 50% of traffic from Penguin 2 and I'm trying to find out why.  My website only has about 150 posts.  According to the Moz Campaign Link Analysys, I have 168 followed external links and more than 11,000 internal links, which I find to be odd.  I thought that may be hurting my rankings

I've read a bunch of articles on Penguin about spam and my website had less than 1% of those types of links.  Most of my links are from relevant websites.  The only link that I think might have hurt me is from a PR4 relevant website that had me on their blogroll.  Over 50% of the links to my site came from just that one website because I was on their blogroll.  But this still doesn't make sense to me because that link was 100% natural.

[Carla_Dawson]

I agree with Alice I would not block them. If the content is unique, they can't hurt you. Category sections show up an SERP's all the time and will increase your CTR (click thru rate)

[evolvingSEO]

Penguin 2.0 is definitely from webspam and backlinks. I suggest using Google Webmaster Tools as your backlink dataset. They provide links that are more likely to be causing problems. It's extremely unlikely the internal links are hurting you if it is Penguin 2.0.

[evolvingSEO]

I agree with David here. Someone made a custom category plugin for WordPress which will do what David is suggesting. I would not block anything, but rather noindex what you don't want in the index, which is typically for me tags, subpages, author and date archives.

-Dan

[uesat]

I used Google Webmaster tools and I used Site Explorer to try and find the spam.  From Webmaster tools was where I found that 50% of all my backlinks came from just 1 site.  But this also happens to be the strongest relevant site I could get backlinks from.  Is this something that Penguin penalizes you for?  The fact that most links are coming from just 1 website?

[evolvingSEO]

It depends. If the sitewide links were;

• off topic from the site it was on, not as relevant
• among a giant list of other "blogroll" links
• on a site that may have a penalty against it
• exact match commercial anchor text <--big one
• if it's part of a "link exchange"

Those issues could hurt you coming from one site. If the links were exact match commercial anchors I'd see about changing them to brand or domain name anchors. If the site linking to you is a random topic, or you're among a big giant list of links, or part of a link exchange get the links removed or disavow.

-Dan

[ALLee]

agree with Dan. If it's not any of the above and you are sure none of the other links are spam, you might not have been hit with penguin 2.0, but some other penalty.

I would check with other tools as well. Use as many tools you have at your disposal and create a comprehensive list of backlinks.

[uesat]

Dan,

The website's topic is very similar to mine.  The link was part of a blogroll of about 15 other sites.  The anchor text was my website name.  For example, if my website were www.cleanbluecars.com, the anchor was Clean Blue Cars.  Maybe Google is considering that to be an exact match commercial anchor text?

The websites on that blogroll were not part of a link exchange, but some of the websites on that blogroll mention the main website on their blog.  The main website hosts a blog carnival every Monday so people naturally mention it.  The main website was who put me on its blogroll because I normally participate on their weekly blog carnival.  The carnival links are and have always been no-follow.

[uesat]

Alicia,

I was hit by penguin 2.0.  There's no question about it

[evolvingSEO]

Thanks for the details. It may not be those links, but if there were even a few others, it could potentially hurt the site. I had a client who had done some spammy link building in 2011ish with another agency and just those few links hurt the site. I would try removing or disavowing anything you can find that's spammy, and not helping, and see if they helps anything.

The types of things to de-optimize on your own site would be more like keyword stuffing in the titles, URLs, headers and internal anchors - not so much blocking pages from being crawled. Blocking crawling it typically only if you want to improve crawl efficiency on really large sites or you just don't want Google to page attention to that content, but it won't fix a Penguin issue if it's on-site.

-Dan

[uesat]

Thanks Dan.  Just to be clear, what do you mean by this "It may not be those links, but if there were even a few others, it could potentially hurt the site."  You mean blogroll links in other sites?  I have 82 links from Woorank.  I did not ask them to put me there and I didnt even know who they were until I saw them on webmaster tools.  They're unrelated to my niche but they're not a bad quality site.  Could they hurt my site?

I did have spammy unrelated links from 3 other websites.  But that's it.  I have a few links from general topic websites and most of my links are from relevant sites.  I'm also on another blogroll from a relevant site, just not as big as the one I mentioned.

[evolvingSEO]

Hi, sorry... I just meant if you did have a few other spammy links from other sites, those could hurt you. I doubt the woorank ones are hurting, but perhaps the other 3 spammy sites you mentioned could have been.