Removing duplicated content using only the NOINDEX in large scale (80% of the website).

[DmitriiK]

Good point! News gotta be new

[CleverPhD]

Couple of things here.

1. If a second Panda update has not occurred since the changes that were made then you may not get credit for the noindexed content.   I don't think this is "cheating" as with the noindex, it just told Google to take 350K of its pages out of the index.  The noindex is one of the best ways to get your content out of Google's index.

2. If you have not spent time improving the non-syndicated content then you are missing the more important part and that is to improve the quality of the content that you have.

A side point to consider here, is your crawl budget.  I am assuming that the site still internally links to these 350K pages and so users and bots will go to them and have to process etc.  This is mostly a waste of time.  As all of these pages are out of Google's index thanks to the noindex tag, why not take out all internal links to those pages (i.e. from sitemaps, paginated index pages, menus, internal content) so that you can have the user and Google focus on the quality content that is left over.  I would then also 404/410 all those low quality pages as they are now out of Google's index and not linked internally.  Why maintain the content?

[CleverPhD]

Just seeing the other responses.  Agree with what EGOL mentions.  A content audit would be even better to see if there was any value at all on those pages (GA traffic, links, etc).  Odds are though that there was not any and you already killed all of it with the noindex tag in place.

[Lukas_TheCurious]

HI Dimitrii,

thank you very much for your opinion. The idea of canonical links is very interesting. We may try that in the "first" phase. But I still miss the point of paying for the content that is not accessible from SE.

Is my understanding right, that if I would set canonical for these duplicates, Google has no reason to show this pages in the SERP?

[DmitriiK]

But I still miss the point of paying for the content that is not accessible from SE

• "paying"?

Is my understanding right, that if I would set canonical for these duplicates, Google has no reason to show this pages in the SERP?

• correct

[Lukas_TheCurious]

EGOL your insights are very appreciated :-)!

I agree with you. Makes total sense.

So you didn't experience any problems removing outdated  content (or "content with no traffic value") from your website? You have set 410 for those pages and remove all internal links to them and google was ok with that?

Redirecting useless content - you mean set 301 to the most relevant page that is bringing traffic?

Thank you sir

[Lukas_TheCurious]

Yeah, paying ... we actually pay for this content (earlier management decisions :-))

[DmitriiK]

Yaikes! Will you guys still pay for it if it's removed? If so, then combining below comments with my thoughts - I'd delete it, since it's old and not time relevant.

[EGOL]

We deleted thousands of pages every few months.

Before deleting anything we identified valuable pages that continued to receive traffic from other websites or from search.  These were often updated and kept on the site.  Everything else was 301 redirected to the "news homepage" of the site.  This was not a news site, it was a very active news section on an industry portal site.

You have set 410 for those pages and remove all internal links to them and google was ok with that?

Our goal was to avoid internal links to pages that were going to be deleted.  Our internal "story recommendation" widgets would stop showing links to pages after a certain length of time.  Our periodic purges were done after that length of time.

We never used hard coded links in stories to pages that were subject to being abandoned.  Instead we simply linked to category pages where something relevant would always be found.

Develop a strategy for internal linking that will reduce site maintenance and focus all internal links to pages that are permanently maintained.

[Lukas_TheCurious]

Yeah, this strategy will be definitely part of the guidelines for the editors.

One last question: do you know some good resources I can use as an inspiration?

Thank you so much..

[Lukas_TheCurious]

1. it has been almost a year now from the massive hit. after that, there were also some smaller hits

2. we are putting effort into improvements. that is quite frustrating for me, because I believe that our effort is demolished by old duplicated content (that creates 80% of the website :-))

Yeah, we will need to take care about the link-mess... 
Thank you!