The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. 10,000 New Pages of New Content - Should I Block in Robots.txt?

    10,000 New Pages of New Content - Should I Block in Robots.txt?

    Intermediate & Advanced SEO
    2 2 709
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • cmaddison
      cmaddison last edited by

      I'm almost ready to launch a redesign of a client's website. The new site has over 10,000 new product pages, which contain unique product descriptions, but do feature some similar text to other products throughout the site.

      An example of the page similarities would be the following two products:

      1. Brown leather 2 seat sofa

      2. Brown leather 4 seat corner sofa

      Obviously, the products are different, but the pages feature very similar terms and phrases.

      I'm worried that the Panda update will mean that these pages are sand-boxed and/or penalised.

      Would you block the new pages? Add them gradually? What would you recommend in this situation?

      1 Reply Last reply Reply Quote 0
      • EGOL
        EGOL last edited by

        Consider reversing your thinking from "what will be my loss to panda" into "what can I do to make this site kick ass".

        Reach for opportunity, extend yourself.

        If this was my site I would get a writer on those product descriptions to make them unquestionably unique, beef them up, add salesmanship and optimize them for search.  This will give you substantive unique content, that converts better, pulls more long tail traffic and moves out of competition with other sites that do the minimal.

        Sure, it will cost money but in the long run it could bring back a huge return.

        My only caution on this is that if you make this investment in writing you need to do that on a site that has can pull reasonable traffic.  If you do this on a site that has no links it will not do you much good.  It is part of a marketing plan not a single item on a "to do" list.

        1 Reply Last reply Reply Quote 1
        • 1 / 1
        • First post
          Last post
        • Pages blocked by robots
          mihoreis
          mihoreis
          0
          3
          69

        • "noindex, follow" or "robots.txt" for thin content pages
          khi5
          khi5
          0
          6
          1.5k

        • Pages getting into Google Index, blocked by Robots.txt??
          Devanur-Rafi
          Devanur-Rafi
          0
          10
          673

        • When you add 10.000 pages that have no real intention to rank in the SERP, should you: "follow,noindex" or disallow the whole directory through robots? What is your opinion?
          Zanox
          Zanox
          0
          6
          101

        • Blocking poor quality content areas with robots.txt
          KaneJamison
          KaneJamison
          0
          4
          128

        • E-commerce Adding New Content - Blog vs New Page
          SammyT
          SammyT
          0
          5
          515

        • Why are new pages not being indexed, and old pages (now in robots.txt) remain in the index?
          KeriMorgret
          KeriMorgret
          0
          3
          378

        • What content should I block in wodpress with robots.txt?
          ENSO
          ENSO
          0
          4
          518

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy