When Mobile and Desktop sites have the same page URLs, how should I handle the 'View Desktop Site' link on a mobile site to ensure a smooth crawl?

[merch_zzounds]

We're about to roll out a mobile site. The mobile and desktop URLs are the same.  User Agent determines whether you see the desktop or mobile version of the site.  At the bottom of the page is a 'View Desktop Site' link that will present the desktop version of the site to mobile user agents when clicked.

I'm concerned that when the mobile crawler crawls our site it will crawl both our entire mobile site, then click 'View Desktop Site' and crawl our entire desktop site as well.  Since mobile and desktop URLs are the same, the mobile crawler will end up crawling both mobile and desktop versions of each URL.  Any tips on what we can do to make sure the mobile crawler either doesn't access the desktop site, or that we can let it know what is the mobile version of the page?

We could simply not show the 'View Desktop Site' to the mobile crawler, but I'm interested to hear if others have encountered this issue and have any other recommended ways for handling it.  Thanks!

[RyanPurkey]

When the crawler goes to the desktop site the user agent should change dynamically and it will see that the user agent is set to non-mobile.  Be sure to check it out here: https://www.google.com/webmasters/tools/mobile-friendly/
 while responsive design fundamentals can be found here: https://developers.google.com/web/fundamentals/layouts/rwd-fundamentals/. Cheers!

[DirkC]

Hi,

Google's advice on this configuration is to use the "Vary HTTP header" to let the Google bot know that the same url is used for mobile/desktop. See also: https://developers.google.com/webmasters/mobile-sites/mobile-seo/configurations/dynamic-serving?hl=en . While Google strongly recommends responsive design - this option is also valid. Matt Cuts says the same in this video: https://www.youtube.com/watch?v=va6qtaiZRHg (it starts with all the caching stuff, the part which might be of interest to you starts at 3:38)

Hope this clarifies,

Dirk