The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Tools
    4. Robots.txt file issues on Shopify server

    Robots.txt file issues on Shopify server

    Moz Tools
    2 2 44
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A_Q
      A_Q last edited by

      We have repeated issues with one of our ecommerce sites not being crawled. We receive the following message:

      Our crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster. Read our troubleshooting guide.

      Are you aware of an issue with robots.txt on the Shopify servers? It is happening at least twice a month so it is quite an issue.

      1 Reply Last reply Reply Quote 0
      • Expansyon
        Expansyon last edited by

        I leave you 2 url, but it seems that the robots do not let you modify it.

        They propose you to put pages as non-index....

        https://community.shopify.com/c/Shopify-Discussion/How-can-I-edit-robots-txt/td-p/432485/page/4

        https://help.shopify.com/en/manual/promoting-marketing/seo/hide-a-page-from-search-engines

        greetings

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        • Robots.txt blocking Moz
          ChristinaRadisic
          ChristinaRadisic
          0
          6
          291

        • Shopify crawl issues
          SilentGorilla
          SilentGorilla
          0
          5
          629

        • Magento CE 1.9.1 - Robots.txt help please
          iamgreenminded
          iamgreenminded
          0
          5
          1.4k

        • Moz campaign works around my robots.txt settings
          VinceWicks
          VinceWicks
          0
          4
          149

        • Does Rogerbot respect the robots.txt file for wildcards?
          AC_Pro
          AC_Pro
          0
          4
          574

        • Seomoz bar: No Follow and Robots.txt
          squareplug
          squareplug
          0
          7
          1.3k

        • To block with robots.txt or canonicalize?
          STPseo
          STPseo
          0
          2
          522

        • SEOMoz site crawlers created an issue for our servers
          G-Force
          G-Force
          0
          4
          992

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy