Duplicate content - news archive

[fdmgroup]

Most of them are due to news items having more than 1 category – which is pretty normal.Also /us/blog, /uk/blog and /ca/blog are effectively the same page.None of them are actually duplicate content – just alternate URLs for the same pagehttp://www.fdmgroup.com/category/news/

[TomRayner]

I'm afraid your blog pages are in fact duplicate content, in Google's eyes anyway.

The /us/blog, /uk/blog and /ca/blog examples are all separate URLs that you are asking Google to index (separate canonical tags for each and no robots instructions that I can see).  Google is going to look at these and any blog posts within them as separate pages.  Once it realises they all have the same content, it will likely result in a Panda algorithmic penalty.

The risk here is that this penalty might affect your entire domain, rather than the offending pages.  I really don't see that as a risk worth taking.  Therefore, I strongly advise to remove the separate versions of the blogs and consolidate into one blog, with redirections of the local blogs to the new ones.  Failing that, choose one version and instruct Google not to index other versions of the page by using a meta robots tag in your header, or in the robots.txt file.

I also advise that you noindex the category page to be sure that its content isn't being seen as duplicate either.  More info on how to do that can be found in the Moz Robots Guide.

[fdmgroup]

From my developer:"Doing a bit of research Google have explicitly stated that that don’t penalise duplicate content unless it appears to be deliberately deceptive. The only issue is which version appears in the search results.Duplicate content on a site is not grounds for action on that site unless it appears that the intent of the duplicate content is to be deceptive and manipulate search engine results. If your site suffers from duplicate content issues, and you don't follow the advice listed above, we do a good job of choosing a version of the content to show in our search results.https://support.google.com/webmasters/answer/66359?hl=enMatt Cutts, Google’s head of search spam, posted a video today about duplicate content and the repercussions of it within Google’s search results.Matt said that somewhere between 25% to 30% of the content on the web is duplicative. Of all the web pages and content across the internet, over one-quarter of it is repetitive or duplicative.But Cutts says you don’t have to worry about it. Google doesn’t treat duplicate content as spam. It is true that Google only wants to show one of those pages in their search results, which may feel like a penalty if your content is not chosen — but it is not.Google takes all the duplicates and groups them into a cluster. Then Google will show the best of the results in that cluster.Matt Cutts did say Google does reserve the right to penalize a site that is excessively duplicating content, in a manipulative manner. But overall, duplicate content is normal and not spam.http://searchengineland.com/googles-matt-cutts-25-30-of-the-webs-content-is-duplicate-content-thats-okay-180063http://searchengineland.com/googles-matt-cutts-duplicate-content-wont-hurt-you-unless-it-is-spammy-167459Cheers"

[JaneCopland]

Hi Christopher,

Google has definitely become a lot better in recent years at identifying this sort of duplication and dealing with it (largely because this has to be one of the most common accidental / non-malicious duplication causes - categories on blogs and news sites). That said, cleaning it up is for the best. I have been meaning to clean up a blog belonging to a family member in this manner for months (years...) because the version of each piece of content Google has chosen is wrong. Pages marked up with dates, e.g. example.com/2014/01 kept ranking better than the original posts in that date range. So even when Google makes the decision for you, it won't necessarily make the right one. You're risking visitors coming to a page they didn't expect, or a page that doesn't answer their query as succinctly as the "best" version would have, and if you are in e-commerce of any sort or focusing on conversions, this can make a big difference to how optimised your traffic's on-site experience is.

Where you'll be "penalised" for duplicate content, especially by Panda, is as you cite above: when the duplication looks like it has been done for spam purposes. This has happened accidentally to people when their content management systems have gone mad with infinite duplication, but it likely won't happen with simple blog categories.

In short, Google sees this sort of duplication all day, every day and will choose its favourite version to rank. However, if you can guide its choice, you're in control of what your visitors see.

You mention country-based categories in your original question. If internationalisation and duplicate content are a concern, you might want to check out the href lang tag (also called rel="alternate" tag - it gets called either by the community). Could be useful if you're publishing the same thing in different countries.

[fdmgroup]

From developer: "Looking into this, we need to have /uk/blog, /us/blog and /ca/blog in order for them to appear on the menus – we could put a noindex meta tag on the us and ca pages to avoid duplicates?"

[fdmgroup]

From developer: "Looking into this, we need to have /uk/blog, /us/blog and /ca/blog in order for them to appear on the menus – we could put a noindex meta tag on the us and ca pages to avoid duplicates?"

Or do you recommend href lang tag? Thanks.