I have 15,000 pages. How do I have the Google bot crawl all the pages?

[Ishimoto]

I have 15,000 pages. How do I have the Google bot crawl all the pages? My site is 7 years old. But there are only about 3,500 pages being crawled.

[AdamThompson]

Google will only index your pages if it deems they are "worthy". However, you can certainly give Googlebot some encouragement. A good way to do this is to setup an XML sitemap and submit it via Google Webmaster Tools.

[Ishimoto]

I did set up an XML sitemap and submit it via Google Webmaster Tools. It did not help. Is it because my PR is 2?

[Svetoslav]

Tihs one is interesing. I work with a currency exchange/transfer site where I have 20000+ pages in english only. What I did is pretty basic, but it worked. I did one sitemap for all the main pages - service pages, homepage and pages which won't change until the next redesign. I did one more XML sitemap file, where I had my first set of money transfer pairs grouped - country to country. My 3rd and largest XML file is where I had 16512 currency combinations listed, by each combo being a webpage. For my english version I have 16058 out of them indexed. The pages are quite similar by content, but also by function. I have 4 variables which seems to do the trick. WIth other language targetings my success ranges from 3000-12000 indexed pages from this particular sitemap.

I guess it depends on the market you are targeting. If the pages have similar content it won't hurt doing some alterations to provide custom information if possible.

Hope that helps!

[KeriMorgret]

Can you tell us the URL of the site in question? That can help us to help you, because we can look at the site and maybe spot something like an improper robots.txt or site architecture that makes it hard for a robot to crawl.