The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. How ro write a robots txt file to point to your site map

    How ro write a robots txt file to point to your site map

    Technical SEO Issues
    4 3 8.7k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Nightwing
      Nightwing last edited by

      Good afternoon from still wet & humid wetherby UK...

      I want to write a robots text file that instruct the bots to index everything and give a specific location to the sitemap. The sitemap url is:http://business.leedscityregion.gov.uk/CMSPages/GoogleSiteMap.aspx

      Is this correct:

      User-agent: *
      Disallow:
      SITEMAP: http://business.leedscityregion.gov.uk/CMSPages/GoogleSiteMap.aspx

      Any insight welcome 🙂

      1 Reply Last reply Reply Quote 0
      • RyanKent
        RyanKent last edited by

        Yes, your example is correct.

        A great page for learning about robots.txt is: http://en.wikipedia.org/wiki/Robots_exclusion_standard#Sitemap

        I will share the official method of declaring your sitemap location involves only the first letter being capitalized (i.e. Sitemap not SITEMAP) but I am almost certain it does not make a difference.

        A few other suggestions which are best practices but do not have to be followed:

        • use all lowercase letters in URLs

        • name the sitemap file "sitemap" not "GoogleSiteMap"

        • submit XML sitemaps when possible. I am again almost certain Google can read other versions so if all you care about is Google then it's fine but otherwise I would suggest just using xml files.

        example: business.leedscityregion.gov.uk/cmspages/sitemap.xml

        Some other helpful links:

        http://support.google.com/webmasters/bin/answer.py?hl=en&answer=183668

        http://www.sitemaps.org/protocol.html

        Matthew_Edgar 1 Reply Last reply Reply Quote 2
        • Matthew_Edgar
          Matthew_Edgar @RyanKent last edited by

          Ryan's answer is correct. I just wanted to jump in to say that I know from first hand experience that Google and Bing are both able to read the sitemap file even if it is a different extension and even if you can't name it sitemap.xml.

          Nightwing 1 Reply Last reply Reply Quote 0
          • Nightwing
            Nightwing @Matthew_Edgar last edited by

            Thank you so much for all your replies 🙂
            [CASE CLOSED]

            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            • Blocking subdomains with Robots.txt file
              PaulM01
              PaulM01
              0
              3
              641

            • Multiple robots.txt files on server
              peakdistrictseo
              peakdistrictseo
              0
              8
              2.4k

            • Will it be possible to point diff sitemap to same robots.txt file.
              DirkC
              DirkC
              0
              3
              268

            • Robots.txt file
              Asher
              Asher
              0
              3
              261

            • Do i have my robots.txt file set up properly
              ClaireH-184886
              ClaireH-184886
              1
              4
              319

            • Robots.txt blocking site or not?
              RyanKent
              RyanKent
              0
              2
              445

            • Mobile site: robots.txt best practices
              AlanMosley
              AlanMosley
              0
              2
              542

            • Use of Robots.txt file on a job site
              jennita
              jennita
              0
              5
              850

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy