How to remove duplicate content issues for thin page(containing oops no resulting found)

[surabhi6]

In this scenarios we have multiple different URLs but the page content is rendering same (containing Oops message) due to which content duplicate's issue arises.As soon as content for these URL,s are available then those pages duplicate issue will removed. So we want to remove duplicate issue not the page & Page URLs.

[TomVolpe]

Hi,

If you simply want to stop these pages from being indexed until they have some non-duplicate content on them, you could use the meta robots tag in your

This is how that would look:

<meta name="robots" content="<strong>noindex</strong>"></meta name="robots">

And this will immediately drop these pages from the index when googlebot crawls a page with this directive in the

If you wish to stop these pages being crawled by rogerbot (moz’s crawler) so the duplicate content errors do not show up, you could add these lines to your robots.txt:

User-agent: rogerbot

Disallow: /oopspage1.html

Disallow:/oops/page2.php

User-agent: *
Allow: /

This will still allow googlebot  to crawl these pages, and will keep them in the index, so when you do update the content you won’t have any problems if you forget to remove the meta robots tag on these pages.

Be careful with robots.txt and remember that Disallow:/oops/ will stop crawlers from crawling subpages of the /oops/ folder, so you need to remember your filenames. But in this case, since you’re only setting these rules for rogerbot, it wouldn’t be catastrophic for your site but it’s best to take care either way.

In reality though, duplicate content is a problem when multiple pages are competing with each other for the same terms. Sometimes the wrong version of the page will be displayed above your preferred version. Unless you’re looking to rank for your ‘Oops nothing here’ phrase, these duplicate content problems are nothing to worry about and the notices will soon disappear from your crawl errors when you create some fresh content for them.

Hope this helps,

Tom