Google & Bing not indexing a Joomla Site properly....

[JohnW-UK]

Can someone explain the following to me please.

The background:

I launched a new website - new domain with no history. I added the domain to my Bing webmaster tools account, verified the domain and submitted the XML sitemap at the same time. I added the domain to my Google analytics account and link webmaster tools and verified the domain - I was NOT asked to submit the sitemap or anything. The site has only 10 pages.

The situation:

The site shows up in bing when I search using site:www.domain.com - Pages indexed:- 1 (the home page) The site shows up in google when I search using site:www.domain.com - Pages indexed:- 30 Please note Google found 30 pages - the sitemap and site only has 10 pages - I have found out due to the way the site has been built that there are "hidden" pages i.e. A page displaying half of a page as it is made up using element in Joomla.

My questions:-

1. Why does Bing find 1 page and Google find 30 - surely Bing should at least find the 10 pages of the site as it has the sitemap? (I suspect I know the answer but I want other peoples input).

2. Why does Google find these hidden elements - Whats the best way to sort this - controllnig the htaccess or robots.txt OR have the programmer look into how Joomla works more to stop this happening.

3. Any Joomla experts out there had the same experience with "hidden" pages showing when you type site:www.domain.com into Google.

I will look forward to your input!

[RyanKent]

** Why does Bing find 1 page and Google find 30 **

Bing is much more selective then Google when it comes to indexing a site. Additionally, Bing takes longer as well. That has always been my experience but if others feel differently feel free to share.

Bing does has a way for you to manually submit all 10 pages. From the Bing Dashboard choose CONFIGURE > Submit URL, then enter each URL. By submitting the URL in this manner you can be certain Bing sees all your site's pages.

To be clear, Bing may crawl the page and choose not to index it. Bing also many index a page then later choose to drop it from their index. Bing has high quality standards related to content and various trust factors.

Why does Google find these hidden elements - Whats the best way to sort this - controllnig the htaccess or robots.txt OR have the programmer look into how Joomla works more to stop this happening.

Who built your site? Did you have a "random" developer build it? Or a professional Joomla developer who focuses only on building Joomla sites? How much experience does your developer have with the particular version of Joomla being used (likely 2.5 or 3.0)? Since you did not share your URL, the best I can offer is general advice. Try going using the HTML code validator from W3C. If you see dozens of errors then the site was not cleanly coded and you may have various issues.

I generally do not advice using robots.txt to block elements as they may still be crawled. I would need to view the site to offer more targeted advice.

Any Joomla experts out there had the same experience with "hidden" pages showing when you type site:www.domain.com into Google.

It can easily happen and typically occurs when a developer's focus is delivering the site rather then SEO. A developer's focus is typically satisfying you, their client, which is not unreasonable. Your requests likely focused on the appearance of the site and it's main functionality. It takes a lot more time and effort to developer an SEO optimized site when compared to a "regular" site.

[JohnW-UK]

Thanks Ryan -

1. I thought as much with Bing but wanted to see other people thoughts - I will hunt around for the submit in webmaster tools. It begs the obvious question what's better quality (bing being selective) or quantity (google analysing it all and deciding for its self).... To be debated at length! lol

2 & 3. W3C no errors and no css errors either..... I think it is the way we put the pages together using modules and laying them out via css - we employ our own coder. I don't really want to broadcast clients sites on forums etc.... But I am looking to improve to ensure we are doing things right - if something is not right we need to do it again and get it right. I don't want to get a rep for bad quality and bad work.