The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Have a client that migrated their site; went live with noindex/nofollow and for last two SEOMoz crawls only getting one page crawled. In contrast, G.A. is crawling all pages. Just wait?

    Have a client that migrated their site; went live with noindex/nofollow and for last two SEOMoz crawls only getting one page crawled. In contrast, G.A. is crawling all pages. Just wait?

    Technical SEO Issues
    5 3 422
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • alankoen123
      alankoen123 last edited by

      Client site is 15 + pages. New site had noindex/nofollow removed prior to last two crawls.

      1 Reply Last reply Reply Quote 0
      • Nobody1560986989723
        Nobody1560986989723 last edited by

        Are you able to post the URL? That way I, or another Mozzer could check the crawlability and just check you're not accidentally blocking robots another way?

        I would also check your robots.txt to make sure you're not Disallowing all your site.

        alankoen123 1 Reply Last reply Reply Quote 0
        • KeriMorgret
          KeriMorgret last edited by

          If GA means Google Analytics, Google Analytics doesn't crawl pages -- it reports visits to pages that have the GA code on them. You can have the content behind a password and blocked in robots.txt and if GA code is there, it would still track people.

          https://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-re-not-crawling-all-my-pages lists some common reasons why only one page is crawled. If you still have problems with crawling, send a note to help@seomoz.org and we'll help you figure things out.

          1 Reply Last reply Reply Quote 0
          • alankoen123
            alankoen123 @Nobody1560986989723 last edited by

            Sure. The site is bayareadrainage.com . Thanks in advance.

            Nobody1560986989723 1 Reply Last reply Reply Quote 0
            • Nobody1560986989723
              Nobody1560986989723 @alankoen123 last edited by

              Okay I just did a test crawl with 'Screaming Frog' SEO spider - no issues

              Robots.txt - not blocking it.

              Headers - not blocking them.

              Therefore definitely look at Keri's post as it's something SEOmoz's end, so check out that thread for some possible answers 🙂

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Very wierd pages. 2900 403 errors in page crawl for a site that only has 140 pages.
                H.M.N.
                H.M.N.
                0
                6
                64

              • Why does my site have so many crawl errors relating to the wordpress login / captcha page
                ZenyaS
                ZenyaS
                0
                6
                88

              • If I want clean up my URLs and take the "www.site.com/page.html" and make it "www.site.com/page" do I need a redirect?
                Booj
                Booj
                0
                4
                113

              • How can I change the page title "two" (artigos/page/2.html) in each category ?
                sprynewmedia
                sprynewmedia
                0
                3
                118

              • Website ranking went from page one to not in top 50 overnight. Help/suggestions?
                Tosten
                Tosten
                0
                5
                280

              • Installing WordPress on a site OR just adding a blog page on the site - Which one is better and why ?
                Personnel_Concept
                Personnel_Concept
                0
                5
                301

              • I have a site that has both http:// and https:// versions indexed, e.g. https://www.homepage.com/ and http://www.homepage.com/. How do I de-index the https// versions without losing the link juice that is going to the https://homepage.com/ pages?
                fthead9
                fthead9
                0
                3
                729

              • If two links from one page link to another, how can I get the second link's anchor text to count?
                RyanKent
                RyanKent
                0
                2
                720

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy