The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Tools
    4. Finding the source of duplicate content URL's

    Finding the source of duplicate content URL's

    Moz Tools
    2 2 300
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DocdataCommerce
      DocdataCommerce last edited by

      We have a website that displays a number of products. The product has variations (sizes) and unfortunately every size has its own URL (for now anyway). Needless to say, this causes duplicate content issues. (And of course, we are looking to change the URL's for our site as soon as possible)

      However, even though these duplicate URL's exist, you should not be able to land on them by navigating through the site. In theory, the site should always display the link to the smallest size. It seems that there is a flaw in our system somewhere, as these links are now found in our campaign here on SEOmoz.

      My question: is there any way to find the crawl path that lead to the URL's that shouldn't have been found, so we can locate the problem?

      1 Reply Last reply Reply Quote 0
      • Mark_Ginsberg
        Mark_Ginsberg last edited by

        Using the Screaming Frog SEO Spider (free version to download will crawl 500 URLs, paid version [99 GBP for a yearly license] will crawl as much as you want), you can see all of the inlinks to a particular page. So run a crawl of the site, you should find those pages with Screaming Frog, and then you can view the inlinks to those pages. Visit the inlinks, and check the code for the links to the page you're looking for - this will quickly show you where the links are to the pages you're trying to hide.

        Also, have you checked the sitemap - the CMS might create links to these pages in the sitemap.

        good luck and let me know if you need any more help with this.

        Mark

        1 Reply Last reply Reply Quote 1
        • 1 / 1
        • First post
          Last post
        • Missing/Duplicate Content but it's definitely all right!
          BlueprintMarketing
          BlueprintMarketing
          0
          3
          55

        • Large site with content silo's - best practice for deep indexing silo content
          1
          1
          159

        • Duplicate Content - Multiple URL's
          Andy.Drinkwater
          Andy.Drinkwater
          0
          5
          132

        • Solving 'duplicate content' for page 2 of X for 1 blog post
          DanielMulderNL
          DanielMulderNL
          0
          3
          877

        • On Page URL's not updating?
          RobertFisher
          RobertFisher
          0
          2
          125

        • Can't find duplicate page content
          KeriMorgret
          KeriMorgret
          0
          4
          349

        • URL paramters and duplicate content
          yacpro13
          yacpro13
          0
          3
          330

        • Which would you chose? Link on PA56 with 88 OBL's and 80 IBL's or a link on a PA75 with 225 OBL's & 40 IBL (Same Domain)
          Dan-Petrovic
          Dan-Petrovic
          0
          5
          988

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy