Ajax4SEO and rogerbot crawling

[LeahHutcheon]

Has anyone had any experience with seo4ajax.com and moz?

The idea is that it points a bot to a html version of an ajax page (sounds good) without the need for ugly urls. However, I don't know how this will work with rogerbot and whether moz can crawl this. There's a section to add in specific user agents and I've added "rogerbot".

Does anyone know if this will work or not? Otherwise, it's going to create some complications. I can't currently check as the site is in development and the dev version is noindexed currently.

Thanks!

[SamWeber]

Hi Philip!

This question is a bit intricate.

With AJAX content like this, I know Google's full specifications

https://developers.google.com/webmasters/ajax-crawling/docs/specification

indicate that the #! and ?escaped_fragment= technique works for their crawlers. However, Roger is a bit picky and isn't robust enough yet to use only the sitemap as the reference in this case. Luckily, one of our wonderful users came up with a solution using pushState() method. Click here:

http://www.moz.com/blog/create-crawlable-link-friendly-ajax-websites-using-pushstate

to find out how to create crawl-able content using pushState. The only other thing I can think of is to run a crawl test if the site is live. You'll have to remove the noindex tag, but updating the robots.txt to allow rogerbot but have a wildcard disallow for the other crawlers should still keep the site from being indexed.

Hopefully this will help!

Best,

Sam
Moz Helpster