The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Use of Robots.txt file on a job site

    Use of Robots.txt file on a job site

    Technical SEO Issues
    5 3 850
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • WebTalent
      WebTalent last edited by

      We are performing SEO on a large niche Job Board. My question revolves around the thought of no following all the actual job postings from their clients as they only last for 30 to 60 days. Anybody have any idea on the best way to handle this?

      1 Reply Last reply Reply Quote 0
      • malecce
        malecce last edited by

        I do not know if I understand correctly
        Do you want  insert no following to all the job posting that expire in 60 days?
        If it 's so, you can put a control  in the cms for the date of expiry of the job posting

        If  somebody  click on the offer expired by SERP, you can retrieve a little script  with a 301 redirect to the job posting similar category to the expired.Ciao

        1 Reply Last reply Reply Quote 0
        • jennita
          jennita last edited by

          Hi Oliver!

          Before coming to SEOmoz I used to work for OntargetJobs which is a company that has multiple niche job boards. Here's what I would recommend:

          1. Keep those pages followed because people will link to them and you want to preserve as much of the link equity as you possibly can. So how do you do that?
          2. Make sure that when a job expires (or gets removed, whatever) that the page gets 301 redirected to the category page the job is posted under. Depending on the niche, it may be locale based, in that case redirect it to the location. The idea here is to send the user to a helpful page for good user experience and conserve some link equity at the same time.
          3. On the page that gets redirected to, program it so when a redirection happens that it displays a message at the top of the page. Something along the lines of "Oops! The job you were looking for is no longer active. However here are similar jobs in XYZ category"

          Again as I mentioned above, this is a good way to help user experience, plus keep some of that link equity from the inevitable links job posting pages get.

          I hope this helps!

          Jen

          WebTalent 1 Reply Last reply Reply Quote 1
          • WebTalent
            WebTalent @jennita last edited by

            Thanks Jennifer! Great answer - I wasn't sure if which strategy would be better. Your answer makes a lot of sense. Thanks for your input!

            jennita 1 Reply Last reply Reply Quote 0
            • jennita
              jennita @WebTalent last edited by

              Happy to help!

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Blocking subdomains with Robots.txt file
                PaulM01
                PaulM01
                0
                3
                641

              • Should you use robots.txt for pages within your site which do not have high quality content or are not contributing a great deal so when Google crawls your site the best performing content has a higher chance of being indexed?
                Jacksons_Fencing
                Jacksons_Fencing
                0
                5
                44

              • Multiple robots.txt files on server
                peakdistrictseo
                peakdistrictseo
                0
                8
                2.4k

              • Meta Robots Noindex and Robots.txt File
                Devanur-Rafi
                Devanur-Rafi
                0
                2
                125

              • Robots.txt & Mobile Site
                LynnPatchett
                LynnPatchett
                0
                3
                4.1k

              • Robots.txt file
                Asher
                Asher
                0
                3
                261

              • Is my robots.txt file working?
                Resultify
                Resultify
                0
                5
                1.1k

              • Mobile site: robots.txt best practices
                AlanMosley
                AlanMosley
                0
                2
                542

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy