How long does it take for Google to index a Sitemap? Should I be worried?

[Travis-W]

Hi forum, I submitted my sitemap to Google 4 days ago, and it has only indexed 4,896 out of 61,367 URLs submitted. I know my sitemap does not contain any errors. How long do you expect it will take Google to index my sitemap? What causes Google to not want to index some of my pages? Thanks!

[DarinPirkey]

In my experience, on average Google will only index about 80% of your sitemap.  Do you already have links to those pages or is this a new site?   Generally, I give it two full weeks to see where it is at and see if any errors come back.

Are you using WMT to verify submission or site:operator?

As far as time goes.  Make sure that your site's crawl rate is optimized too.  GoogleBot works on a kind of time "budget" and can't crawl everything all the time.

Check for Dup Content.  Google doesn't "hit" your site but will choose one version or another to index if the content is similar or nearly similar.

These are some starting points but 4 days isn't very long for that many pages.  Give it some time and keep us posted on how it's going.

[irvingw]

if done properly google should accept all of your urls or something close to that number. Four days is not a lot of time o give them though.

you said you submitted your sitemap (singular) - 61,367 is too many URLs per sitemap, Google says no more than 50,000 i would split it in 3rds or 4ths though if you have 61,000+ urls.

Have you manually groomed your sitemap to make sure there are no urls with parameters or blocked sections or anything like that?

[Travis-W]

Hi Irving,

It is actually a sitemap index containing 2 sitemaps, one with 40k URLs (to stay under both the file size and URL amount limits) and another with 21,367.

Also, what do you mean when you say 'parameters?'

[Travis-W]

Google seems to be indexing approximately 20 pages per day since the initial jump.  How can I optimize my crawl rate?

I submitted my sitemap through Google Webmaster Tools.

[DarinPirkey]

Google doesn't want to take up to much bandwidth on your server so it will "budget" its time during a crawl.  Make sure your page load times are fast, server has a good "up time", constantly add new content so when Google does the next crawl it will see that it is a site that is constantly updated, and make sure your pages aren't too large.  You can actually speed up Google's crawl (or slow it down) under configuration>settings> but you don't need to do anything here but make sure that the radial that says "Let Google Decide" is selected under crawl rate.

If you haven't done so already, you can Fetch as Google and submit to index with all linked pages.  Technically Google sees your site so you don't have to do this but I always see an increase in the crawl rate when I do it.  (Don't do it if you've already done it recently).  I also like to submit individual pages to the Fetch as Google when I create new content and I only index that particular page if it's a brand new page that I've added.

When your PR increases, you'll also see an increase to the crawl rate.