The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Can Google Crawl This Page?

    Can Google Crawl This Page?

    Technical SEO Issues
    4 2 263
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • shr109
      shr109 last edited by

      I'm going to have to post the page in question which i'd rather not do but I have permission from the client to do so.

      Question: A recruitment client of mine had their website build on a proprietary platform by a so-called recruitment specialist agency.  Unfortunately the site is not performing well in the organic listings.

      I believe the culprit is this page and others like it: http://www.prospect-health.com/Jobs/?st=0&o3=973&s=1&o4=1215&sortdir=desc&displayinstance=Advanced Search_Site1&pagesize=50000&page=1&o1=255&sortby=CreationDate&o2=260&ij=0

      Basically as soon as you deviate from the top level pages you land on pages that have database-query URLs like this one.  My take on it is that Google cannot crawl these pages and is therefore having trouble picking up all of the job listings.  I have taken some measures to combat this and obviously we have an xml sitemap in place but it seems the pages that Google finds via the XML feed are not performing because there is no obvious flow of 'link juice' to them.

      There are a number of latest jobs listed on top level pages like this one: http://www.prospect-health.com/optometry-jobs and when they are picked up they perform Ok in the SERPs, which is the biggest clue to the problem outlined above.

      The agency in question have an SEO department who dispute the problem and their proposed solution is to create more content and build more links (genius!).

      Just looking for some clarification from you guys if you don't mind?

      1 Reply Last reply Reply Quote 0
      • N1ghteyes
        N1ghteyes last edited by

        I think your pretty much spot on. Google -can- crawl queries but they wont rank very well at all.

        Your best bet will be to change: (just reading into this that looks like a 'get all jobs' query)

        http://www.prospect-health.com/Jobs/?st=0&o3=973&s=1&o4=1215&sortdir=desc&displayinstance=Advanced Search_Site1&pagesize=50000&page=1&o1=255&sortby=CreationDate&o2=260&ij=0

        to just

        http://www.prospect-health.com/Jobs

        There are loads of ways to remove the query string and keep the functionality, depending on the software powering the site, personally i'd fix up the search to POST data so that it keeps the url clean and add in the appropriate routes to create the path.

        I have some experience of job search sites (having worked on a couple of the largest in the UK) and breaking URLS down something like this seems to work best. (depending on the data you have obviously)

        <domain>/jobs/<location>/</location></domain>

        You could also take a look at how other job sites structure their URLS, (monster.co.uk, targetjobs.co.uk, jobsite.co.uk etc)

        Let me know if you need a hand and i'll see if i can be more specific. (you'll have to tell me what its running on though)

        EDIT:: You should force lower-case on urls as well. Caps wont effect google but they arn't user friendly (miss types etc)

        1 Reply Last reply Reply Quote 3
        • shr109
          shr109 last edited by

          Thanks Toby, good to get a second opinion on these things and some clarification.

          The platform is the agency's own proprietary one but i don't know if it's based on an existing framework or completely bespoke. Having looked at some of the other sites they have build though, it seems other clients are experiencing similar indexing problems as they have all utilised a workaround of some sort.

          I'll share the name of the agency with you by email if you want to do some digging but I don't think it's fair to name and shame them on here - shr109@hotmail.com

          1 Reply Last reply Reply Quote 0
          • N1ghteyes
            N1ghteyes last edited by

            Hi shr109,

            I've sent an email over so you have my address. Please let me know if it doesnt come through, we're recovering from a couple of email issues this end (infected web server in the same IP Subnet as our email server got us blacklisted), it might have ended up in spam!

            Thanks,

            1 Reply Last reply Reply Quote 1
            • 1 / 1
            • First post
              Last post
            • How can i make Google to consider my News pages
              AlexisWithers
              AlexisWithers
              0
              4
              301

            • Page disappeared from Google index. Google cache shows page is being redirected.
              shop.nordstrom
              shop.nordstrom
              0
              5
              761

            • Can Googlebot crawl the content on this page?
              danatanseo
              danatanseo
              0
              7
              156

            • I have 3500 pages crawled by Google, - why is SEOMOZ only able to crawl 400 of these ?
              Abe_Schmidt
              Abe_Schmidt
              0
              2
              95

            • Can Google show the hReview-Aggregate microformat in the SERPs on a product page if the reviews themselves are on a separate page?
              0
              1
              1.0k

            • How do I keep Google from crawling my PPC landing page?
              chris.kent
              chris.kent
              0
              3
              422

            • Have a client that migrated their site; went live with noindex/nofollow and for last two SEOMoz crawls only getting one page crawled. In contrast, G.A. is crawling all pages. Just wait?
              Nobody1560986989723
              Nobody1560986989723
              0
              5
              422

            • How can I get Google to crawl my site daily?
              labradoodlelocator
              labradoodlelocator
              0
              7
              4.2k

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy