How a google bot sees your site

[StreetwiseReports]

So I have stumbled across various websites like this:

http://www.smart-it-consulting.com/internet/google/googlebot-spoofer/

The concept here is to be able to view your site as a googlebot sees it. However, the results are a little puzzling. Google is reading the text on my page but not the title tags according to the results. Are websites like this accurate OR does Google not read title tags and H1 tags anymore?

Also on a slighly related note. I noticed the results show the navigation bar is being read first by google, is this bad and should the navigation bar be optimized for keywords as well? If it did, it would read a bit funny and the "humans" would be confused.

[DougRoberts]

I don't know how far I'd trust such third party sites. Take a look at Google Webmaster Tools. There's a Fetch as Googlebot tool under diagnostics.

Here's the Google help about it::

Fetch as Googlebot

Google certainly does read title and heading tags!

As far as the navigation bar goes - always think about humans first. Sometimes you can improve relevance by avoiding generic names. Avoid generic terms like "Articles" for instance and replace it with something that better describes the content behind the click "nutrition guide" or "food facts"...

The fact that your navigation is being read first isn't a problem - it's a convention that is hardly going to be penalised by a search engine.

[kdaly100]

A lot of these sites are badly coded garbage.  I would ignore these sites.

[rhutchings]

These sites are good for a quick scan of the contextual formatting of a website, but not for really telling how Google (or any specific search engine) sees your site. That specific one you linked to is horrible.

Google does see title, H1, other headers, meta description and most elements of your site.  A more accurate way to see how google sees your site would be to:

1. See how the page looks in the index.  Type  "site:myspecificurl.com" into google for the page you want to see and google will just return the results of what it has in its index.  That is how google sees your site. If your site/pages are not in the index, get them in (#2 below).
2. Verify through Google Webmaster Tools.  In the webmaster tools you can see what pages of your site are being indexed/crawled through google, and you can also request specific pages to be crawled again if you need.  This combined with an xml sitemap will usually get pages indexed pretty quick, and then you can verify with the same methodology as i mentioned above.
3. Use the SEOmoz pro toolset here and set up a campaign and the tool will tell you if you are missing any title tags or other important on-page elements.  the seomoz "bot" crawls similar to google, so that should give you a feel for how it works.

[StreetwiseReports]

Thanks, but even under that Fetch as Googlebot link you posted, I don't see how to get an accurate tool of how Google views your site.

[DougRoberts]

The instructions are near the bottom of the page:

In order to use Fetch as Googlebot, you'll need to have added and verified your site in Webmaster Tools. Then, follow these instructions:

1. On the Webmaster Tools Home page, click the site you want.
2. On the Dashboard, under Diagnostics, click Fetch as Googlebot.
3. In the text box, type the path to the page you want to check.
4. In the dropdown list, select the type of fetch you want. To see what our web crawler Googlebot sees, select Web. To see what our mobile crawler Googlebot-Mobile sees, select cHTML (this is used mainly for Japanese web sites) or Mobile XHTML/WML.
5. Click Fetch.

Once googlebot has fetched your page you'll have a "success" link that you can click on to see what Googlebot saw.

This will be the header, including the server response code and then the html that googlebot received.

What this doesn't tell you is how this was interpreted by Google, of course this is where SEOMoz's on-page reports and crawl stats can help detect errors and way your can improve your on-page optimisation.

[SearchDexSEO]

You need to pull this forum question.  That link redirects to a spammy site about "Freeing Syria."