The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Should I disallow crawl of my Job board?

    Should I disallow crawl of my Job board?

    Technical SEO Issues
    3 3 81
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • O2C
      O2C last edited by

      MOZ crawler is telling me we have loads of duplicate content issues. We use a Job Board plugin on our Wordpress site and we have allot of duplicate or very similar jobs (usually just a different location), but the plugin doesn't allow us to add any rel canonical tags to the individual jobs.

      Should I disallow the /jobs/ url in the robots.txt file? This will solve the duplicate content issue but then Google wont be able to crawl any of the individual job listings

      Has anyone had any experience working with a job board plugin on Wordpress and had a similar issue, or can advise on how best to solve our duplicate content??

      Thanks 🙂

      1 Reply Last reply Reply Quote 0
      • evolvingSEO
        evolvingSEO last edited by

        Hi David

        You can probably leave the pages as-is and allow Google to crawl them. But you may want to update the part of the content that's triggering the duplicate errors. In other words - are your title tags and meta descriptions unique for each page? Or maybe the H1's are duplicates? Since the pages do have slight differences, I would use those differences to make the content unique.

        1 Reply Last reply Reply Quote 1
        • MattRoney
          MattRoney last edited by

          Hi David! Did Dan's answer help? Let us know if there's anything else we can do to help you work this out. 🙂

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Google Crawling Issues! How Can I Get Google to Crawl My Website Regularly?
            libero_net
            libero_net
            1
            8
            142

          • Crawl issues
            Christy-Correll
            Christy-Correll
            0
            3
            105

          • Disallow statement - is this tiny anomaly enough to render Disallow invalid?
            lzhao
            lzhao
            0
            3
            118

          • Crawl Results
            sherohass
            sherohass
            0
            5
            132

          • How to force a crawl
            KeriMorgret
            KeriMorgret
            0
            2
            356

          • Crawl rate
            jwdl
            jwdl
            0
            5
            658

          • Have a client that migrated their site; went live with noindex/nofollow and for last two SEOMoz crawls only getting one page crawled. In contrast, G.A. is crawling all pages. Just wait?
            Nobody1560986989723
            Nobody1560986989723
            0
            5
            422

          • Site Crawl
            Sean_Dawes
            Sean_Dawes
            0
            2
            414

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy