The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Other Research Tools
    4. Keyword Grader & OmniExplorer_Bot Question

    Keyword Grader & OmniExplorer_Bot Question

    Other Research Tools
    5 3 637
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • OOMDODigital
      OOMDODigital last edited by

      I found this in several of my clients Robots.txt

      "User-Agent: OmniExplorer_Bot

      Disallow: /"
      

      Right now I cannot grade any of them that are on a particular platform that has that written in.

      Would this block the keyword grader?

      Thanks!

      1 Reply Last reply Reply Quote 0
      • MikeRoberts
        MikeRoberts last edited by

        Moz's crawler is rogerbot so that shouldn't block it as far as I know. What does the rest of your robots.txt look like? Maybe something in the format of it is accidentally disallowing more bots than you meant to block.

        1 Reply Last reply Reply Quote 0
        • JoelDay
          JoelDay last edited by

          Hey,

          Mike is correct that our crawler is named RogerBot.

          I took a look at one of your campaigns that was only crawling one and it appears that it's returning an access denied page.

          This is likely something based on the server itself for RogerBot since it looks like it's returning full source to other User-Agents.
          I hope that helps track down the issue. 
          Cheers,
          Joel.

          OOMDODigital 1 Reply Last reply Reply Quote 2
          • OOMDODigital
            OOMDODigital last edited by

            
            User-Agent: bingbot
            
            

            Crawl-delay: 5

            User-Agent: msnbot
            
            

            Crawl-delay: 5

            User-Agent: Googlebot-Mobile
            
            

            Allow: /mobi

            User-Agent: OmniExplorer_Bot
            
            

            Disallow: /

            1 Reply Last reply Reply Quote 0
            • OOMDODigital
              OOMDODigital @JoelDay last edited by

              I'm going to reach out to our vendor to try and resolve the issue. In your experience, what would be the best way to explain this problem to their tech team and try to resolve it?

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Keyword Explorer search tips&tricks e.g. A&B , A&B Not C, AorB
                tawnycase
                tawnycase
                0
                2
                46

              • Moz Pro Question: Does the amount of keywords you are allowed to search reset each month?
                ChiarynMiranda
                ChiarynMiranda
                0
                2
                163

              • Onpage Grader not Finding Keyword in URL
                DavidLee
                DavidLee
                0
                5
                145

              • Q&A Question showing as answered when it is not
                TheWebMastercom
                TheWebMastercom
                0
                5
                122

              • How Does the On-Page Grader Know about my Target Keyword? Or how can I tell it?
                Titan552
                Titan552
                0
                6
                72

              • On-Page Grader is not counting H2 and H3 Keywords
                Guilhermealvesc
                Guilhermealvesc
                0
                3
                194

              • Keyword Ranking Question - Paid Ads?
                Symmetri
                Symmetri
                0
                3
                116

              • On Page Grader Question
                JoelDay
                JoelDay
                0
                2
                84

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy