The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Google is Still Blocking Pages Unblocked 1 Month ago in Robots

    Google is Still Blocking Pages Unblocked 1 Month ago in Robots

    Technical SEO Issues
    3 2 131
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Tyler123
      Tyler123 last edited by

      I manage a large site over 200K indexed pages. We recently added a new vertical to the site that was 20K pages. We initially blocked the pages using Robots.txt while we were developing/testing. We unblocked the pages 1 month ago. The pages are still not indexed at this point. 1 page will show up in the index with an omitted results link. Upon clicking the link you can see the remaining un-indexed pages. Looking for some suggestions. Thanks.

      1 Reply Last reply Reply Quote 0
      • DirkC
        DirkC last edited by

        Hi,

        Fetch the main page(s) with "Fetch as Google" under the Crawl section in Webmaster tools - then submit to the index.

        You are sure that there are no other elements blocking the indexing of the page (like meta tag  or X-robots tags in the header?

        Also fetch the new robots.txt file - to be sure that Google notices that it has changed.

        Did you add a sitemap for this new section - does it show any notifications/warnings in WMT?

        rgds,

        Dirk

        Tyler123 1 Reply Last reply Reply Quote 2
        • Tyler123
          Tyler123 @DirkC last edited by

          Thanks. I fetched both the main page and made a slight tweak to the robots and resubmitted last night. It looks like it is making a bit of progress. There is nothing else blocking the pages. We did add a new sitemap when we first launched the pages with no warnings. I did notice yesterday that we had approx 268 broken links that went to 404 pages in WMT in this specific sub-folder. The discovery of the broken link-404 pages by Google seems to be around the same time Google stopped crawling this section of the site. We took care of the broken links this morning. Thanks for the help!

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • How to get google to forget my old but still working page and list my new fully optimized page for a keyword?
            Neckermann
            Neckermann
            0
            3
            200

          • Blog Page Titles - Page 1, Page 2 etc.
            Paul-Gross
            Paul-Gross
            0
            8
            549

          • Do I need to block my cart page in robots.txt?
            Hutch42
            Hutch42
            0
            3
            2.4k

          • 3,511 Pages Indexed and 3,331 Pages Blocked by Robots
            PeaSoupDigital
            PeaSoupDigital
            0
            4
            114

          • Should I block Map pages with robots.txt?
            imaginex
            imaginex
            0
            3
            73

          • Page disappeared from Google index. Google cache shows page is being redirected.
            shop.nordstrom
            shop.nordstrom
            0
            5
            761

          • Google indexing despite robots.txt block
            john4math
            john4math
            0
            13
            654

          • Cached pages still showing on Google
            aspenchicago
            aspenchicago
            0
            4
            175

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy