The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Would spiders successfully crawl a page with two distinct sets of content?

    Would spiders successfully crawl a page with two distinct sets of content?

    Intermediate & Advanced SEO
    4 4 147
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ClayPotCreative
      ClayPotCreative last edited by

      Hello all and thank you in advance for the help.

      I have a coffee company that sell both retail and wholesale products. These are typically the same product, just at different prices. We are planning on having a pop up for users to help them self identify upon their first visit asking if they are retail or wholesale clients.

      So if someone clicks retail, the cookie will show them retail pricing throughout the site and vice versa for those that identify themselves as wholesale. I can talk to our programmer to find out how he actually plans on doing this from a technical standpoint if it would be of assistance.

      My question is, how will a spider crawl this site? I am assuming (probably incorrectly) that whatever the "default" selection is (for example, right now now people see retail pricing and then opt into wholesale) will be the information/pricing that they index.

      So long story short, how would a spider crawl a page that has two sets of distinct pricing information displayed based on user self identification? Thanks again!

      1 Reply Last reply Reply Quote 0
      • AarcMediaGroup
        AarcMediaGroup last edited by

        In most platforms we work with, the "retail" pricing and descriptions are all the spiders see, because the wholesale pricing is hidden from the general public and is only presented when a wholesale customer "logs into the site".

        To pitch the "wholesale side" you would simply optimize a page or pages for the terms relating to wholesale coffee you want to rank for.  There would be no need for the "wholesale prices or info" to be crawled in that case.  Only the pages pitching your wholesale program would get crawled.

        Now many make the mistake of creating two separate sites, and in that case, you run the risk of them competing against each other which is bad for rankings.

        1 Reply Last reply Reply Quote 1
        • MoosaHemani
          MoosaHemani last edited by

          In my personal opinion, crawler crawl through URLs so if the URL is not changing upon price change then in that case crawler will crawl the prices that are available as default. Adding different URLs for wholesalers will get you in to the duplicate content issues (in-case you do not create completely separate content).

          As far as I know price change might not help with rankings but content does so let the retail page ranking and focus on helping wholesalers to find their area through user friendly visit experience.

          Hope this helps!

          1 Reply Last reply Reply Quote 1
          • Everett
            Everett last edited by

            You are assuming correctly. The spider will see the content that is shown by default, which in your case should be the retail pricing.

            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            • Will google be able to crawl all of the pages given that the pages displayed or the info on a page varies according to the city of a user?
              0
              3
              17

            • Pages with Duplicate Page Content (with and without www)
              Andy.Drinkwater
              Andy.Drinkwater
              0
              3
              181

            • Duplicate Page Content Issues Reported in Moz Crawl Report
              dan.lecocq
              dan.lecocq
              0
              2
              253

            • Interlinking from unique content page to limited content page
              khi5
              khi5
              0
              9
              100

            • Duplicate content within sections of a page but not full page duplicate content
              J_Sinclair
              J_Sinclair
              0
              3
              112

            • Does Google crawl and spider for other links in rel=canonical pages?
              SEOAndy
              SEOAndy
              0
              3
              1.1k

            • Are links to on-page content crawled / have any effect on page rank?
              Vahe.Arabian
              Vahe.Arabian
              0
              4
              320

            • 202 error page set in robots.txt versus using crawl-able 404 error
              EricaMcGillivray
              EricaMcGillivray
              0
              3
              870

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy