The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Same URL in "Duplicate Content" and "Blocked by robots.txt"?

    Same URL in "Duplicate Content" and "Blocked by robots.txt"?

    Technical SEO Issues
    3 2 502
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • alsvik
      alsvik last edited by

      How can the same URL show up in Seomoz Crawl Diagnostics "Most common errors and warnings" in both the "Duplicate Content"-list and the "Blocked by robots.txt"-list?

      Shouldnt the latter exclude it from the first list?

      1 Reply Last reply Reply Quote 0
      • ChrisCountey
        ChrisCountey last edited by

        I think that both of these tools are mutually exclusive. It does not take into consideration that a page blocked to spiders won't count against you in terms of duplicate content, but maybe it should!

        alsvik 1 Reply Last reply Reply Quote 0
        • alsvik
          alsvik @ChrisCountey last edited by

          It's kinda strange. I would expect all URLs blocked by robots.txt to NOT show up anywhere on Seomoz.

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Google Webmaster Tools is saying "Sitemap contains urls which are blocked by robots.txt" after Https move...
            vetofunk
            vetofunk
            0
            5
            11.2k

          • What are the negative implications of listing URLs in a sitemap that are then blocked in the robots.txt?
            Dezzign
            Dezzign
            0
            3
            153

          • When is Duplicate Content Duplicate Content
            AMHC
            AMHC
            0
            6
            169

          • Google is indexing blocked content in robots.txt
            bjs2010
            bjs2010
            0
            5
            147

          • "Extremely high number of URLs" warning for robots.txt blocked pages
            KristinaKledzik
            KristinaKledzik
            0
            8
            380

          • IIS 7.5 - Duplicate Content and Totally Wrong robot.txt
            jesse-landry
            jesse-landry
            0
            3
            305

          • Blocked URL's by robots.txt
            meralucian37
            meralucian37
            0
            7
            398

          • Block Quotes and Citations for duplicate content
            vforvinnie
            vforvinnie
            1
            4
            1.7k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy