The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Crawl diagnostic issue?

    Crawl diagnostic issue?

    Intermediate & Advanced SEO
    6 3 112
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • nettt
      nettt last edited by

      I'am sorry if my English isn't very good, but this is my problem at the moment:

      On two of my campagnes I get a weird error on Moz Analytics:

      605 Page Banned by robots.txt, X-Robots-Tag HTTP Header, or Meta Robots Tag

      Moz Analytics points to an url that starts with: http:/**/None/**www.????.com. We don't understand how Moz indexed this non-existing page that starts with None? And how can we solve this error?

      I hope that someone can help me.

      1 Reply Last reply Reply Quote 0
      • GPainter
        GPainter last edited by

        Is the URL correct on Moz pro?  It also seems like your robots.txt is blocking Moz which you may want to look into.

        1 Reply Last reply Reply Quote 1
        • nettt
          nettt last edited by

          Thanks for the respons, Pixelbypixel!

          I have checked the URL, and it is not our own website that has the error.

          We have checked the robots.txt and it should not cause any problem. We have n't recently changed it.

          I Think that Moz is causing it, but I am not sure..

          GPainter 1 Reply Last reply Reply Quote 1
          • GPainter
            GPainter @nettt last edited by

            "I have checked the URL, and it is not our own website that has the error."

            is this the problem?

            Could you take a screen grab of the problem it might help better.

            1 Reply Last reply Reply Quote 1
            • DavidLee
              DavidLee last edited by

              Hi Nettt!

              I apologize for any confusion and can confirm there is no issue on your side. One of our crawlers failed causing some campaigns crawled on Aug 29th attempt to follow the strange /None/ URL you are seeing in your diagnostics. I've submitted a re-crawl for all of your campaigns affected so you should see updated data by this Friday.

              Hope this helps!

              nettt 1 Reply Last reply Reply Quote 1
              • nettt
                nettt @DavidLee last edited by

                Hi MOZ,

                I'am sorry that I have not previously responded. The problem has been solved. Thanks!

                Also thanks to Pixel for the response!

                Greetz,

                Sam

                1 Reply Last reply Reply Quote 1
                • 1 / 1
                • First post
                  Last post
                • I have a metadata issue. My site crawl is coming back with missing descriptions, but all of the pages look like site tags (i.e. /blog/?_sft_tag=call-routing)
                  Rajesh.Prajapati
                  Rajesh.Prajapati
                  0
                  2
                  43

                • Post migration issues - #11 + configuration issue
                  0
                  1
                  136

                • Would you rate-control Googlebot? How much crawling is too much crawling?
                  CraigBradford
                  CraigBradford
                  0
                  4
                  807

                • Magento E-Commerce Crawl Issues
                  webmethod
                  webmethod
                  0
                  2
                  949

                • Crawl Issue Found: No rel="canonical" Tags
                  David-Kley
                  David-Kley
                  0
                  4
                  1.1k

                • Hey guys i have this issues on my crawling report what should i do to exlude the pages? are d
                  adulter
                  adulter
                  0
                  5
                  99

                • Issues with Google-Bot crawl vs. Roger-Bot
                  IOSC
                  IOSC
                  0
                  3
                  319

                • Crawl Diagnostics - Understanding Rel Canonical Notices
                  travis.hickox
                  travis.hickox
                  0
                  7
                  484

                Get started with Moz Pro!

                Unlock the power of advanced SEO tools and data-driven insights.

                Start my free trial
                Products
                • Moz Pro
                • Moz Local
                • Moz API
                • Moz Data
                • STAT
                • Product Updates
                Moz Solutions
                • SMB Solutions
                • Agency Solutions
                • Enterprise Solutions
                • Digital Marketers
                Free SEO Tools
                • Domain Authority Checker
                • Link Explorer
                • Keyword Explorer
                • Competitive Research
                • Brand Authority Checker
                • Local Citation Checker
                • MozBar Extension
                • MozCast
                Resources
                • Blog
                • SEO Learning Center
                • Help Hub
                • Beginner's Guide to SEO
                • How-to Guides
                • Moz Academy
                • API Docs
                About Moz
                • About
                • Team
                • Careers
                • Contact
                Why Moz
                • Case Studies
                • Testimonials
                Get Involved
                • Become an Affiliate
                • MozCon
                • Webinars
                • Practical Marketer Series
                • MozPod
                Connect with us

                Contact the Help team

                Join our newsletter
                Moz logo
                © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                • Accessibility
                • Terms of Use
                • Privacy