Big discrepancies between pages in Google's index and pages in sitemap

[Digirank]

Hi,

I'm noticing a huge difference in the number of pages in Googles index (using 'site:' search) versus the number of pages indexed by Google in Webmaster tools. (ie 20,600 in 'site:' search vs 5,100 submitted via the dynamic sitemap.)

Anyone know possible causes for this and how i can fix?

It's an ecommerce site but i can't see any issues with duplicate content - they employ a very good canonical tag strategy. Could it be that Google has decided to ignore the canonical tag?

Any help appreciated,

Karen

[Humix]

Are you using just one sitemap or multiple?

[Digirank]

Just 1 at the moment.

[Humix]

Sorry wrong interpretation of your question. Have you excluded the site search pages using robots.txt.? If not this might be the reason why you've that many pages indexed.

Anyway this discussion might give you more answers:

[Digirank]

Hi,

Thanks so much for that, it's really interesting.

I've resolved the issue now, it was a case of some (a lot!) of missing canonical tags. Phew!

Thanks for your help!

[David-Kley]

Take a look at the pages that are indexed. Chances are that since it is a cart or CMS-based site, you just need to use robots.txt to block out some areas you don't want indexed. You also need to look at your indexed pages, to see if any of them are duplicates, meaning you have 2 or more url's that display the same content.

"It's an ecommerce site but i can't see any issues with duplicate content - they employ a very good canonical tag strategy. Could it be that Google has decided to ignore the canonical tag? "

Could be that your cms or cart is not forwarding all the pages to the canonical version. Again, check to see if you can access multiple versions of the same page. Ecom and CMS sites always have these types of errors if you dont keep a close eye on the URL's since they are database driven, vs static HTML. Look for www or non-www versions of pages, url's with and without index.php, etc.

Once you target what the offending url's are, use redirects to forward them to the proper and search engine friendly version.