The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. A site is not being indexed by Google Yahoo or Bing

    A site is not being indexed by Google Yahoo or Bing

    Technical SEO Issues
    6 3 539
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Intergen
      Intergen last edited by

      This site - http://adoptionconnection.org/ is not being indexed by any of the search engines.  I checked the easy stuff - robots text is:

        <meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">all, index, follow</a>" />  <meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">noodp</a>" />  <meta name="<a class="attribute-value">robots</a>" content="<a class="attribute-value">noydir</a>" /> I have checked what I can determine would cause the issue but have found nothing
      to prevent it from being indexed.
      
      I'm thinking it may be re-directs etc. Any answer would be great.
      
      Thanks in advance,
      
      1 Reply Last reply Reply Quote 0
      • Saijo.George
        Saijo.George last edited by

        You could make sure the relevant content is listed in your xml site map for starters : http://adoptionconnection.org/sitemap.xml ( its currently blank, assuming of course you have submitted it to Google via Webmasters tools )

        Jump in to Web Masters Tools and see the crawl stats and see if there are any errors

        Intergen 1 Reply Last reply Reply Quote 0
        • john4math
          john4math last edited by

          The robots.txt (at http://adoptionconnection.org/robots.txt) is disallowing all bots from reading the site.  Change it from:

          User-agent: *
          Disallow: /
          

          to:

          User-agent: *
          Disallow:
          
          Intergen Saijo.George 2 Replies Last reply Reply Quote 3
          • Intergen
            Intergen @Saijo.George last edited by

            Great answer thanks I will make sure they have submitted to webmaster tools.

            1 Reply Last reply Reply Quote 0
            • Intergen
              Intergen @john4math last edited by

              I think this is it John!  I will have them make the change tomorrow & get back on this thread to give the results.

              Thanks in advance!!

              1 Reply Last reply Reply Quote 0
              • Saijo.George
                Saijo.George @john4math last edited by

                There lies the problem , best to manual add the admin folder and other folder you dont want spiders accessing in there

                This should help : http://www.robotstxt.org/robotstxt.html

                1 Reply Last reply Reply Quote 1
                • 1 / 1
                • First post
                  Last post
                • Should you use robots.txt for pages within your site which do not have high quality content or are not contributing a great deal so when Google crawls your site the best performing content has a higher chance of being indexed?
                  Jacksons_Fencing
                  Jacksons_Fencing
                  0
                  5
                  44

                • Dropped out of Bing index!! (and Yahoo too)
                  corlin
                  corlin
                  0
                  7
                  1.1k

                • Why is Google not indexing my site?
                  Shawn_Huber
                  Shawn_Huber
                  0
                  18
                  198

                • Bing is not Indexing my site.
                  saurabh1905
                  saurabh1905
                  0
                  5
                  1.9k

                • Dev Site Was Indexed By Google
                  customerparadigm.com
                  customerparadigm.com
                  0
                  5
                  166

                • Site not ranking in Google but comes up #1 in Yahoo and Bing
                  SEO5Team
                  SEO5Team
                  0
                  7
                  4.1k

                • Site being indexed by Google before it has launched
                  josh-riley
                  josh-riley
                  0
                  4
                  698

                • Site being indexed by Google before it has launched
                  KeriMorgret
                  KeriMorgret
                  0
                  3
                  246

                Get started with Moz Pro!

                Unlock the power of advanced SEO tools and data-driven insights.

                Start my free trial
                Products
                • Moz Pro
                • Moz Local
                • Moz API
                • Moz Data
                • STAT
                • Product Updates
                Moz Solutions
                • SMB Solutions
                • Agency Solutions
                • Enterprise Solutions
                • Digital Marketers
                Free SEO Tools
                • Domain Authority Checker
                • Link Explorer
                • Keyword Explorer
                • Competitive Research
                • Brand Authority Checker
                • Local Citation Checker
                • MozBar Extension
                • MozCast
                Resources
                • Blog
                • SEO Learning Center
                • Help Hub
                • Beginner's Guide to SEO
                • How-to Guides
                • Moz Academy
                • API Docs
                About Moz
                • About
                • Team
                • Careers
                • Contact
                Why Moz
                • Case Studies
                • Testimonials
                Get Involved
                • Become an Affiliate
                • MozCon
                • Webinars
                • Practical Marketer Series
                • MozPod
                Connect with us

                Contact the Help team

                Join our newsletter
                Moz logo
                © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                • Accessibility
                • Terms of Use
                • Privacy