Many Duplicate Content Flags

[HZseo]

Not sure about you all, but I’m loving the new Moz Site Crawler.  However, I was noticing that it is identifying a huge amount of pages as duplicate content.

There are about 30,000 pages in this website, with that said we’ve had to make many templates to make the site scalable.  Additionally a url rule was lost which caused a significant amount of duplicate pages to be created.  I am working through using the moz crawl tool to identify duplicate pages but noticing many pages under “Affected Pages,” are actually unique content pages with initial content that is duplicate.

I read that Moz flags any pages with 90% or more content overlapping content or code.  My theory for this is that some templates that are too similar, to the point that Moz reads them as duplicative.  Has this happened for anyone else?

In addition, if Moz is flagging these similar pages as duplicate content, do we surmise that Google bots are having the same issue? We have seen issues with rankings as it pertains to the actual duplicate pages but hadn't experienced issues across the unique pages, they are hyperlocal pages so we are able to see rankings quite easily.

[samantha.chapman]

Hey there,

Sam from Moz's Help Team here!

You're correct - our tool has a 90% tolerance for duplicate content, which means it will flag any content that has 90% of the same code between pages. This includes all the source code on the page and not just the viewable text.

You can run your own checks for percentage similarity using this tool: http://smallseotools.com/similar-page-checker/.

If we're identifying two pages as having duplicate content it's likely that Google will be running into the same issues. You can read a little more about duplicate content, and how to resolve it, on our resource page here: https://moz.com/learn/seo/duplicate-content

Let us know if we can help with anything else!