The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Getting Google to index new pages

    Getting Google to index new pages

    Technical SEO Issues
    5 3 886
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • scanlin
      scanlin last edited by

      I have a site, called SiteB that has 200 pages of new, unique content. I made a table of contents (TOC) page on SiteB that points to about 50 pages of SiteB content. I would like to get SiteB's TOC page crawled and indexed by Google, as well as all the pages it points to. I submitted the TOC to Pingler 24 hours ago and from the logs I see the Googlebot visited the TOC page but it did not crawl any of the 50 pages that are linked to from the TOC.

      I do not have a robots.txt file on SiteB. There are no robot meta tags (nofollow, noindex). There are no 'rel=nofollow' attributes on the links.

      Why would Google crawl the TOC (when I Pinglered it) but not crawl any of the links on that page?

      One other fact, and I don't know if this matters, but SiteB lives on a subdomain and the URLs contain numbers, like this:

      http://subdomain.domain.com/category/34404

      Yes, I know that the number part is suboptimal from an SEO point of view. I'm working on that, too. But first wanted to figure out why Google isn't crawling the TOC. The site is new and so hasn't been penalized by Google.

      Thanks for any ideas...

      1 Reply Last reply Reply Quote 0
      • itrogers
        itrogers last edited by

        Have you created and submitted to Google an XML sitemap including the TOC and all other pages?

        scanlin 1 Reply Last reply Reply Quote 0
        • scanlin
          scanlin @itrogers last edited by

          I haven't tried an xml sitemap; I will try that. Thanks for the response...

          KeriMorgret scanlin 2 Replies Last reply Reply Quote 0
          • KeriMorgret
            KeriMorgret @scanlin last edited by

            What are the results from the sitemap? Has Google started crawling any more of your pages?

            1 Reply Last reply Reply Quote 0
            • scanlin
              scanlin @scanlin last edited by

              Turns out I didn't need the sitemap. I just needed more time. Google crawled and indexed everything as I expected, but it took longer than I expected.

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Over 40+ pages have been removed from the indexed and this page has been selected as the google preferred canonical.
                willcritchlow
                willcritchlow
                0
                4
                69

              • How to get google to forget my old but still working page and list my new fully optimized page for a keyword?
                Neckermann
                Neckermann
                0
                3
                200

              • Get List Of All Indexed Google Pages
                UmarKhan
                UmarKhan
                0
                2
                194

              • Is there a way to get Google to index more of your pages for SEO ranking?
                CarlosZambrana
                CarlosZambrana
                0
                5
                2.1k

              • Anything new if determining how many of a sites pages are in Google's supplemental index vs the main index?
                SEMPassion
                SEMPassion
                0
                4
                390

              • Page disappeared from Google index. Google cache shows page is being redirected.
                shop.nordstrom
                shop.nordstrom
                0
                5
                761

              • How do I get google to index the right pages with the right key word?
                ursalesguru
                ursalesguru
                0
                5
                196

              • Changed cms - google indexes old and new pages
                Tit
                Tit
                0
                4
                403

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy