Mobile search results show desktop crawled content

[friendoffood]

This post is deleted!

[KristinaKledzik]

Hi Ted,

I think I may have found the issue: your "Switch to Mobile View" link has a rel="nofollow" on it, so while Google is probably finding and accurately reading the page, you've asked it not to index it.

Just remove the "nofollow" attribute and you should be fine.

Best,

Kristina

[friendoffood]

Thanks Kristina, but I need some clarification:

I thought Google would be indexing www.qjamba.com  and www.qjamba.com/?siteview=mobile as two different links.  Am I wrong about that?

Also, I thought "nofollow" is different than 'noindex', and just tells the desktop bot not to go to the mobile page, because once it does that they program treats it like a mobile device/mobile bot and renders mobile pages.

thanks, Ted

[KristinaKledzik]

Google needs a way to find the mobile version of the page - either through a link on your site (like the link I recommended you change to "follow"), a rel="alternate" tag, or a link from another site. Google crawls nofollowed links, but it doesn't add a page after a nofollowed link to the index if it's not already in there.

Best recommendation is to use the rel="alternate" tag, but I figured the easier answer was to "follow" the link you've already got in there.

Good luck,

Kristina

[friendoffood]

Your experience far exceeds mine, so I don't have much confidence in my response here but I'm a bit puzzled: If I tell the desktop bot to follow to the mobile site, then it will crawl the mobile friendly pages, which I don't want it to do because its the wrong bot to be crawling the mobile pages with.  I assume the mobile bot will find my site the same way the desktop bot will find it, and isn't dependent on the desktop bot to find it.  It's the same url for both: www.qjamba.com  so as long as the mobile bot goes to that url it will crawl it.

From what I just read the alternate tag is used when the page is actually a different url name, and that isn't the case for my site

HERE'S MY CONCERN: The mobile bot has crawled my mobile friendly home page, and it lists it as 'mobile-friendly' on a mobile device, but it is still using the wrong title, which makes me suspect something very worrisome: Google doesn't keep(for anybody) separate indexes for mobile and desktop bots when the url is the same:  It may crawl both and check for mobile rendering for the mobile, and it may use the crawl to determine what other pages to crawl for the respective bot, but the content itself perhaps isn't being stored seperately for the mobile bot (ask yourself -- would it store separate content for mobile when a responsive site design is used?  i'm thinking maybe it wouldn't).

Unfortunately, very few people in SEO seem to know the answer to this critical issue, and it seems very hard to test because of the lag time Google has in putting things into the index (ie it may crawl with the other bot before it puts the content into the index for the first bot--see next para).

IF THAT FEAR IS REALITY, I need to probably move to using a separate domain for my site because my mobile version excludes a whole bunch of content that exists for desktop only--yet if I tell the mobile bot that those URLS are 404 it perhaps would remove the url from the only index it has - the desktop index - and if I tell it they are 301s, then it may replace the desktop index content with the redirection content (the mobile home page), which would totally counterproductive.

Thanks again...I'm really not sure what to do anymore though..

.

[KristinaKledzik]

Ah, sorry about that, since your link on your desktop site has a parameter, I assumed that you were setting up a separate mobile site (which is what you referred to, with a different subdomain) rather than a dynamically served site, which is what you actually have.

Which means you're absolutely right, you don't need links between the two pages.

In that case, I think you're okay. I just double checked on Amazon.com, which is probably the most famous, most powerful dynamically served site in the world. Its mobile version of the homepage has the code:

<title>Amazon.com</title>

But, mobile search results for "amazon.com" return:

Amazon.com: Online Shopping for Electronics, Apparel, Computers ...

Online retailer of books, mobies, music and games along with electronics, toys, apparel, sports, tools, groceries, and...

That's the page title and meta description of the desktop version of the site.

I'll admit, I've never seen Google admit to using the desktop version of the page title and meta description before. But, if Google's doing it to Amazon - whose site we know Google crawls and understands - I think you're okay. Like you said, you have the "mobile friendly" designation, so my guess is, this is a Google decision, not a mistake on your part.

I hope that eases your worries a bit! Really, I don't think there's anything you need to do here. And, I really appreciate you bringing this into the community. I learned something today!

Best,

Kristina

[friendoffood]

Yes and no   First I'm sorry if my wording threw you off on my setup.  These things are so hard sometimes to clearly describe (for me at least).

As for Amazon, it DOES seem to validate the idea that Google is using only one index of content(desktop), which is what I just read at this site, in the comments by Sp4cecat: http://www.sitepoint.com/responsive-design-vs-m-sites.  So for matching urls with only reformatted content, that seems to say I'm on track--no problem....

But here is what may be a big problem for me, I think: Amazon probably is serving all the same urls for both, where I'm serving only some of the desktop pages for mobile.  So, if Google really only is keeping one index (the one for desktop), then I may be throwing them off by giving them a 404 when their mobile bot tries to go to a page that only exists for the desktop.  They may be removing that page from the 'one and only' index they have: desktop.  Then when the desktop bot crawls that url -- it gets put back into the index.  So back and forth it goes, and that can't be a good thing for SEO, can it?  And yet I can't figure out a way to test to see if that's really what happens because of the lag time between crawling and showing up (or removing) from the index!  Arrrgghh..

If all that is true, I have a real problem, as it would suggest that for seo purposes Google expects mobile content to be equivilent to non-mobile content.

Any thoughts on that issue?

[KristinaKledzik]

I think that you're making a pretty big assumption by thinking that Google only has one index. Everything that Google has said and SEOs have seen - the fact that there's a separate Googlebot for mobile, the "mobile friendly" designation, and different results on mobile search than desktop - indicate that Google has a separate index, or at least modifies its desktop index for search.

The only thing that we've found so far is that Google uses the desktop page title and meta description for mobile results. So, I assume that Google is pulling meta data from the desktop crawl to display in mobile search results. It doesn't mean that there isn't a mobile index.

That said, as I responded to your separate question, returning a 404 if you're using a mobile device is not a good idea. Not because of a single index, but because this isn't standard practice, and I have no idea how Google will interpret that. It could lower mobile searches unnecessarily or cause some sort of a cloaking penalty. I doubt it'll pull it in and out of the index.

The best method here is to just let mobile visitors access desktop versions of the page where there isn't a mobile version. Like you mentioned, you could add a pop up inviting them to the mobile version of the site, which will send them to the mobile homepage, but that will probably increase your bounce rate dramatically. You'll have to test a few options to find the best one.

Good luck!

Kristina

[friendoffood]

Thanks very much Kristin.  I have to think about it more..I did just notice when I fetch in Google for desktop it crawls for only the desktop, but when I fetch for mobile, it crawls with both the smartphone and desktop bots.....I'll have to test more..will report any findings if I can figure it out!  the popup was meant for when mobile users click on a non-mobile friendly page--it automatically directs them to the mobile site, but in this case since the page isn't mobile friendly they would get a message just to basically warn them that that PAGE isn't optimized for mobile--sounds like 'best practice' is to give them that page as opposed to a 404 page with options only for mobile pages..

[KristinaKledzik]

Good luck!

[friendoffood]

Hi Kristina,

I just found this:

http://blog.hubspot.com/insiders/how-to-follow-googles-mobile-rules-for-awesome-seo

<<Keep in mind that Google does not have a separate mobile index, so when a user searches from a mobile device, the results are from the same index as they are from a desktop. When a search is executed from a mobile device, Google serves up the desktop version of the URL, and when the searcher clicks a result, the mobile version of the URL is loaded. The way in which the mobile version is loaded is determined by the mobile configuration you have set up. >>

This, if true, validates what I have been suspecting, -- it appears that Google expects basically equivalent content for mobile-friendly and desktop when sharing urls--it doesn't expect a 'minimal' mobile version. I assume Google does this in part because it saves billions of dollars of storage to have only one index--and in the vast majority of cases mobile content SHOULD be equivalent anyway.

It also addressed my other concerns, which validate what you were saying:

<<redirecting mobile="" users="" to="" the="" homepage="" in="" absence="" of="" a="" version="" desktop="" page="" is="" bad="" practice.="" if="" does="" not="" exist,="" should="" be="" displayed.="" additionally,="" there="" with="" no="" equivalent,="" displayed="" for="" desktops.="">></redirecting>

I still plan to run some simple tests with fetching and diff keywords for mobile and desktop pages to verify.  Then will decide what to do.  Thanks very much.

[KristinaKledzik]

Hi again,

I think you may be reading too much into this.

Google used to have separate mobile indexes, and I do remember something about them possibly combining them (although I can't find any articles on that now), but that does not mean that Google doesn't crawl pages that are dynamically served, or recognize that they're different from desktop versions of pages.

I know that your site is partially dynamically served and partially desktop-only, but luckily Google crawls by page, rather than by site. It will be able to figure out which of your pages are mobile optimize and which aren't. It's true that 404ing for mobile agents may mess with that, which is part of why that isn't a good idea. (The other is that it's pretty bad user experience. If someone bookmarks a desktop link to your site, then tries to access it on their phone, if they get a 404 they're going to think your site is broken!)

You're obviously free to keep investigating, but I'm worried you're going to go in circles with all of the partial information out there. Google's pretty smart; build a good, clean site and it'll be able to read it.

Best,

Kristina

[friendoffood]

Thanks.  I just feel like I have to understand it better before deciding what to do.  Adding in the desktop-only pages on a mobile device is likely to create speed issues (as they are currently designed) which will hurt my ranking.  That's one reason I prefer to not have them if it doesn't hurt me in Google's eyes, but i've just about decided something is probably better than nothing as far as SEO is concerned.

That article seems to be clearly saying that there is one index, but it isn't saying they don't crawl both--I'm sure they do, otherwise they wouldn't have both for fetching, but what I think they are doing is crawling to see if the mobile page is mobile friendly--so they can list (and rank it in search results) it as such on mobile devices.  But, since with their  fetch for mobile they immediately crawl desktop too-- it seems to validate that they are using the desktop version to get the actual content for storing in the index tables.

[KristinaKledzik]

Hi there,

Just checking in to see if you figured this one out?

I have an alternative theory: Google may be pulling the page title and meta description from Dmoz for both the mobile and the desktop site. That's what I was actually seeing with Amazon. If this is true, then the page title is not indicating that Googlebot mobile is crawling the desktop version of the site (which is good!).

Best,

Kristina