The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Other Research Tools
    4. Rogerbot will not crawl my site! Site URL is https but keep getting and error that homepage (http) can not be accessed. I set up a second campaign to alter the target url to the newer https version but still getting the same error! What can I do?

    Rogerbot will not crawl my site! Site URL is https but keep getting and error that homepage (http) can not be accessed. I set up a second campaign to alter the target url to the newer https version but still getting the same error! What can I do?

    Other Research Tools
    3 3 200
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • digitalascend
      digitalascend last edited by

      Site URL is https but keep getting and error that homepage (http://www.flogas.co.uk/) can not be accessed. I set up a second campaign to alter the target url to the newer https://www.flogas.co.uk/ version but still getting the same error! What can I do? I want to use Moz for everything rather than continuing to use a separate auditing tool!

      1 Reply Last reply Reply Quote 0
      • Mobilio
        Mobilio last edited by

        I just check and see that http is 301 to https. But https works.

        Can you mail this issue to help@moz.com?

        1 Reply Last reply Reply Quote 1
        • DirkC
          DirkC last edited by

          It will probably help if you delete the following line from your robots.txt file:

          User-Agent: rogerbot
          Disallow: /

          Dirk

          PS You also block /js/ for all bots - Google doesn't really recommend that

          1 Reply Last reply Reply Quote 3
          • 1 / 1
          • First post
            Last post
          • I have too many tittle tag issue for my site on moz site crawl error
            Vanderlindemedia
            Vanderlindemedia
            0
            2
            33

          • What data we don't get from link explorer that we can get if we add a campaign?
            eli.myers
            eli.myers
            0
            2
            39

          • Is there a way to export all your crawl errors for multiple Moz campaigns at once?
            tawnycase
            tawnycase
            0
            2
            75

          • Http:// https:// google search console crawl errors
            O.D.
            O.D.
            0
            5
            355

          • How Do I Troubleshoot 804 HTTPS Crawl Error?
            DirkC
            DirkC
            0
            8
            3.5k

          • We Launched a new site and Rogerbot is still reporting on links/errors from the old site, is there a way to clear those out?
            Abe_Schmidt
            Abe_Schmidt
            0
            2
            58

          • On Page Grader can't access my URLs
            ChiarynMiranda
            ChiarynMiranda
            0
            4
            333

          • How do I get Moz to crawl my site?
            Modi
            Modi
            0
            3
            1.8k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy