Finding Duplicate Content Spanning more than one Site?

[Travis-W]

Hi forum, SEOMoz's crawler identifies duplicate content within your own site, which is great. How can I compare my site to another site to see if they share "duplicate content?" Thanks!

[danatanseo]

Hi Zora,

This best way to do this is to grab a random section of text from the page and go to Google, then paste that section of text in the search bar inside "quotes."  For example, from your question above, I could search:

"SEOMoz's crawler identifies duplicate content within your own site, which is great. How can I compare my site"

you will see that the result in Google is a result to this page (once it's been indexed, which hasn't happened quite yet) - Just be sure to put quotes around your snippet.

Hope that helps!

Dana

[Travis-W]

This helps, thanks Dana. Is the actual paragraph content the main source of a duplicate content penalty? For example, what if the pages share different metadata and the HTML is entirely different except for the H1 text and paragraph content?

[DarinPirkey]

Just for the record there isn't any "Duplicate Content Penalty" so don't worry to much about this. Duplicate content on a site is not grounds for action on that site unless it appears that the intent of the duplicate content is to be deceptive and manipulate search engine results.

However, to answer your question I use copyscape to do this but you have to insert a URL and not just lines at a time.

Here are some other ones I've heard good things about:

DupliChecker

Plagium

Plagiarisma

I agree with Dana on the Google thing too.  Like she said, "Just be sure to put quotes around your snippet."

[danatanseo]

IMHO, even if the HTML is different you could have duplicate content if the H1 or paragraph text is substantially similar. However, is this automatically penalized? No. Syndication of content can be quite prevalent on the Web. For example the AP breaks a news story and posts it online and it is subsequently picked up by the New York Times and Wall Street Journal. Wherever the content appeared first, particularly if it has a canonical tag in place, that source will be credited with having the original content. The other sites aren't going to be penalized, but they aren't going to benefit from it either.

Similar things happen on large e-commerce sites all the time. For example, 100's of e-commerce stores sell lightbulbs. Those descriptions are most certainly "substantially similar." It'd be kind of strange if they weren't. They aren't penalized for that.

I hope this is helpful! It is always good to set up a Google Alert for any great pieces of content you do write, just so you can be aware of who might be copying your stuff! (Tynt.com can also be very useful for this).

Good luck!

Dana

[danatanseo]

Yes. I totally agree with Darin. There isn't a duplicate content penalty, per se, and the tools he listed are quite good suggestions as well.

[DarinPirkey]

The Alert thing is great!  I use it when we write new content (along with CopyScape after a week or so) just so I can make sure I'm outranking it.  lol