The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Other Research Tools
    4. Keyword Grader & OmniExplorer_Bot Question

    Keyword Grader & OmniExplorer_Bot Question

    Other Research Tools
    5 3 637
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • OOMDODigital
      OOMDODigital last edited by

      I found this in several of my clients Robots.txt

      "User-Agent: OmniExplorer_Bot

      Disallow: /"
      

      Right now I cannot grade any of them that are on a particular platform that has that written in.

      Would this block the keyword grader?

      Thanks!

      1 Reply Last reply Reply Quote 0
      • MikeRoberts
        MikeRoberts last edited by

        Moz's crawler is rogerbot so that shouldn't block it as far as I know. What does the rest of your robots.txt look like? Maybe something in the format of it is accidentally disallowing more bots than you meant to block.

        1 Reply Last reply Reply Quote 0
        • JoelDay
          JoelDay last edited by

          Hey,

          Mike is correct that our crawler is named RogerBot.

          I took a look at one of your campaigns that was only crawling one and it appears that it's returning an access denied page.

          This is likely something based on the server itself for RogerBot since it looks like it's returning full source to other User-Agents.
          I hope that helps track down the issue. 
          Cheers,
          Joel.

          OOMDODigital 1 Reply Last reply Reply Quote 2
          • OOMDODigital
            OOMDODigital last edited by

            
            User-Agent: bingbot
            
            

            Crawl-delay: 5

            User-Agent: msnbot
            
            

            Crawl-delay: 5

            User-Agent: Googlebot-Mobile
            
            

            Allow: /mobi

            User-Agent: OmniExplorer_Bot
            
            

            Disallow: /

            1 Reply Last reply Reply Quote 0
            • OOMDODigital
              OOMDODigital @JoelDay last edited by

              I'm going to reach out to our vendor to try and resolve the issue. In your experience, what would be the best way to explain this problem to their tech team and try to resolve it?

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Should keywords found in a keyword tool (eg. Keyword explorer) be implemented on a page in the exact order?
                marktheshark10
                marktheshark10
                1
                3
                86

              • Keyword Explorer search tips&tricks e.g. A&B , A&B Not C, AorB
                tawnycase
                tawnycase
                0
                2
                46

              • Keyword Lists - Omitting Keywords
                brianglassman
                brianglassman
                0
                3
                84

              • How do you block keywords in On-Page Grader for certain URLs?
                Linda-Vassily
                Linda-Vassily
                0
                2
                67

              • On Page Grader Returning Large # of Keywords
                JoelDay
                JoelDay
                0
                3
                60

              • Keyword Ranking Question - Paid Ads?
                Symmetri
                Symmetri
                0
                3
                116

              • On Page Grader Question
                JoelDay
                JoelDay
                0
                2
                84

              • On-page Grader and Cyrillic keywords
                LynnPatchett
                LynnPatchett
                0
                2
                86

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy