The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Meta No INDEX and Robots - Optimizing Crawl Budget

    Meta No INDEX and Robots - Optimizing Crawl Budget

    Intermediate & Advanced SEO
    2 2 419
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bjs2010
      bjs2010 last edited by

      Hi,

      Sometime ago, a few thousand pages got into Google's index - they were "product pop up" pages, exact duplicates of the actual product page but a "quick view".

      So I deleted them via GWT and also put in a Meta No Index on these pop up overlays to stop them being indexed and causing dupe content issues.

      They are no longer within the index as far as I can see, i do a site:www.mydomain.com/ajax and nothing appears -

      So can I block these off now with robots.txt to optimize my crawl budget?

      Thanks

      1 Reply Last reply Reply Quote 0
      • TakeshiYoung
        TakeshiYoung last edited by

        Are you still linking to those pages? If so, I would just keep the noindex,follow meta in the header. That way you still benefit from the link juice flowing to these pages, link juice that would be lost if you just blocked them from being crawled via robots.

        1 Reply Last reply Reply Quote 1
        • 1 / 1
        • First post
          Last post
        • Site moved. Unable to index page : Noindex detected in robots meta tag?!
          effectdigital
          effectdigital
          0
          4
          1.7k

        • Is robots met tag a more reliable than robots.txt at preventing indexing by Google?
          Bobbi_Tschumper
          Bobbi_Tschumper
          1
          7
          3.0k

        • Why is the meta description not the same as in the index?
          Leonie-Kramer
          Leonie-Kramer
          0
          2
          114

        • How can a Page indexed without crawled?
          Devanur-Rafi
          Devanur-Rafi
          0
          7
          91

        • Correct Syntax for Meta No Index
          bjs2010
          bjs2010
          1
          3
          266

        • MOZ crawl report says category pages blocked by meta robots but theyr'e not?
          SamCUK
          SamCUK
          0
          5
          315

        • Why are new pages not being indexed, and old pages (now in robots.txt) remain in the index?
          KeriMorgret
          KeriMorgret
          0
          3
          378

        • Crawl Budget on Noindex Follow
          wojkwasi
          wojkwasi
          0
          2
          927

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy