The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Tools
    4. Finding the source of duplicate content URL's

    Finding the source of duplicate content URL's

    Moz Tools
    2 2 300
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DocdataCommerce
      DocdataCommerce last edited by

      We have a website that displays a number of products. The product has variations (sizes) and unfortunately every size has its own URL (for now anyway). Needless to say, this causes duplicate content issues. (And of course, we are looking to change the URL's for our site as soon as possible)

      However, even though these duplicate URL's exist, you should not be able to land on them by navigating through the site. In theory, the site should always display the link to the smallest size. It seems that there is a flaw in our system somewhere, as these links are now found in our campaign here on SEOmoz.

      My question: is there any way to find the crawl path that lead to the URL's that shouldn't have been found, so we can locate the problem?

      1 Reply Last reply Reply Quote 0
      • Mark_Ginsberg
        Mark_Ginsberg last edited by

        Using the Screaming Frog SEO Spider (free version to download will crawl 500 URLs, paid version [99 GBP for a yearly license] will crawl as much as you want), you can see all of the inlinks to a particular page. So run a crawl of the site, you should find those pages with Screaming Frog, and then you can view the inlinks to those pages. Visit the inlinks, and check the code for the links to the page you're looking for - this will quickly show you where the links are to the pages you're trying to hide.

        Also, have you checked the sitemap - the CMS might create links to these pages in the sitemap.

        good luck and let me know if you need any more help with this.

        Mark

        1 Reply Last reply Reply Quote 1
        • 1 / 1
        • First post
          Last post
        • I have an issue with hubspot's blog platform and duplicate content.
          DarinPirkey
          DarinPirkey
          0
          2
          56

        • Can we discuss Moz's Forecast? It's been over 100 degrees for a month.
          TaylorRHawkins
          TaylorRHawkins
          1
          6
          89

        • 'Duplicate Page Content' for dissimilar pages
          PatrickDelehanty
          PatrickDelehanty
          0
          2
          80

        • Duplicate Content - Multiple URL's
          Andy.Drinkwater
          Andy.Drinkwater
          0
          5
          132

        • Since July 1, we've had a HUGE jump in errors on our weekly crawl. We don't think anything has changed on our website. Has MOZ changed something that would account for a large leap in duplicate content and duplicate title errors?
          KristyFord
          KristyFord
          0
          3
          77

        • Why don't Google+ URL's work in OSE?
          Peterli
          Peterli
          1
          2
          229

        • Can't find duplicate page content
          KeriMorgret
          KeriMorgret
          0
          4
          349

        • Crawl Diagnostics Shows thousands of 302's from a single url. I'm confused
          TheSEODR
          TheSEODR
          0
          4
          247

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy