The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. On-Page / Site Optimization
    4. Should I be disallowing my forum in the robots.txt file?

    Should I be disallowing my forum in the robots.txt file?

    On-Page / Site Optimization
    5 4 287
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bricktech
      bricktech last edited by

      This post is deleted!
      1 Reply Last reply Reply Quote 0
      • eyepaq
        eyepaq last edited by

        Hi,

        Strictly from a technical SEO perspective it's ok to let the forum in the index and let the bots crawl it - for many reasons (eg: internal links, support pages for some keywords and so on).

        Having duplicate Title Tags is not a big issue. The worst negative effect that you can suffer from is for a forum page to rank instead of one of your main landing pages - but that can also be fixed. There are no issues from Google's perspective if you have duplicate Title Tags (again, the only negative effect can be canibalization - you are trying to rank with multiple pages for the same query).

        Now that is from a technical perspective and you should decide if you will leave it "open" or not based on two major factors: the technical aspect but probably more important the content aspect.

        Is the content within the forum good ? High quality ?

        There is an important set of questions (23 to be exact) that you will need to review with the forum:

        http://googlewebmastercentral.blogspot.co.at/2011/05/more-guidance-on-building-high-quality.html

        If at the end if most of the answers are ok - then you should open the forum for the bots - if not, I would keep it  under robots.txt.

        If you have a lot of content and it's at least decent content you can leverage it and gain a lot - after all is UGC and that is gold (if it's not spammy.)

        Hope it helps as a short direction.

        Cheers.

        1 Reply Last reply Reply Quote 2
        • QMRBHAI
          QMRBHAI last edited by

          I have a problem my site's robots.txt is blocking moz can anyone help me

          Regards

          Damon PS2

          1 Reply Last reply Reply Quote 0
          • waleedkhalid
            waleedkhalid last edited by

            Why my site go down after updating Content

            Regards

            homescapes

            homescapesmodapk.com

            1 Reply Last reply Reply Quote 0
            • waleedkhalid
              waleedkhalid last edited by

              I have  a problem my site's robots.txt is blocking moz can anyone help me

              Regarads

              OMOD

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Two Robots.txt files
                TrulyTravel
                TrulyTravel
                0
                5
                514

              • Question about robots.txt
                spencerhjustice
                spencerhjustice
                0
                3
                125

              • How to exclude URL filter searches in robots.txt
                Everett
                Everett
                0
                9
                8.6k

              • Need suggestion: Should the user profile link be disallowed in robots.txt
                KristinaKledzik
                KristinaKledzik
                0
                6
                343

              • Solve duplicate content issues by using robots.txt
                JohnHuynh
                JohnHuynh
                0
                5
                153

              • Robots file include sitemap
                yeagerd
                yeagerd
                0
                2
                180

              • Disallow a spammed sub-page from robots.txt
                zigojacko
                zigojacko
                0
                3
                475

              • Robots.txt file
                Greenman
                Greenman
                0
                3
                652

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy