The pages that add robots as noindex will Crawl and marked as duplicate page content on seo moz ?
-
When we marked a page as noindex with robots like
{<meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">noindex</a>" />}
will crawl and marked as duplicate page content(Its already a duplicate page content within the site. ie, Two links pointing to the same page).So we are mentioning both the links no need to index on SE.But after we made this and crawl reports have no change like it tooks the duplicate with noindex marked pages too.
Please help to solve this problem.
-
What you're seeing is normal. What you want to check is that the number of pages is the number you expect.
From the SEOMoz FAQ on the Crawl Report:
We are still seeing duplicate content on SEOmoz even though we have marked those pages as "noindex, follow." Any ideas why?
SEOMoz is not a search engine index, it uses a crawler. If those pages are not blocked by the robots.txt file, then SEOMoz will crawl them. They ignore the noindex tag because they don't index anything. Search engines will honor the noindex tag and not index a page if you specify with the robots meta tag. However, to remove pages from the crawl, disallow them in the robots.txt or metarobots.(http://www.seomoz.org/help/crawl-diagnostics)
Hope this helps.