The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Other Research Tools
    4. Possible bug in Crawl Issues report?

    Possible bug in Crawl Issues report?

    Other Research Tools
    3 2 269
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A-Drive
      A-Drive last edited by

      Hi all -

      My crawl issues report shows 3 pages with missing titles. These are just google verification files and the robot.txt file - shouldn't these be excluded?

      Pages with Title Missing or Emptyas of May 11
      URL Page Authority Linking Root Domains
      https://www.mysite.com/googlea87e28121c071983.html
      1 0
      https://www.mysite.com/robots.txt
      1 0
      https://www.mysite.com/google9b9dc57478f61677.html

      1 Reply Last reply Reply Quote 0
      • LynnPatchett
        LynnPatchett last edited by

        Hi Matthew,

        I think you will find that those urls are included in your sitemap.xml and that is why they are being crawled. They are also being indexed by google - you can search for the name of the verification files and they show up in the results. Remove them from the sitemap and you should be ok!

        1 Reply Last reply Reply Quote 2
        • A-Drive
          A-Drive last edited by

          Wow that is interesting. Thanks for the info and direction Lynn. I will look into that.

          Matt

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Crawl Issue Question
            K.Net
            K.Net
            0
            4
            37

          • Why is on-demand crawl reporting numbers that are very different than the regualr site crawl by Moz?
            ctripp1010
            ctripp1010
            0
            3
            43

          • Moz Crawl Report Increase in Errors?
            LisaHunt
            LisaHunt
            0
            3
            64

          • Issues with Crawl Test and SSL Certificate
            KristinaKeyser
            KristinaKeyser
            0
            4
            351

          • Crawl Report Internal Links Count
            AISEO
            AISEO
            0
            3
            618

          • Correcting a 4xx on my crawl report
            LynnPatchett
            LynnPatchett
            0
            2
            39

          • Issues with on-page report
            IguanaFix
            IguanaFix
            0
            4
            96

          • Where to find one off crawl report
            RikkiD22
            RikkiD22
            0
            5
            163

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy