After Server Migration - Crawling Gets slow and Dynamic Pages wherein Content changes are not getting Updated

[Modi]

Hello,

I have just performed doing server migration 2 days back

All's well with traffic moved to new servers

But somehow - it seems that w.r.t previous host that on submitting a new article - it was getting indexed in minutes. Now even after submitting page for indexing - its taking bit of time in coming to Search Engines and some pages wherein content is daily updated - despite submitting for indexing - changes are not getting reflected

Site name is - http://www.mycarhelpline.com

Have checked in robots, meta tags, url structure - all remains well intact. No unknown errors reports through Google webmaster

Could someone advise - is it normal - due to name server and ip address change and expect to correct it automatically or am i missing something

Kindly advise in . Thanks

[vivekrathore]

It should not be. check your page load time. If pages takes longer to load than google bot may bounce off. Check your webmaster tool as see if there are any server errors showing.

[Modi]

@Anirban

Thanks, no errors in GWT Tools. Loading time - could not observe a change. As per Gtmetrics - tested - is well within limits. Pages with dynamic content are not getting updated in Search Engine - which earlier was happening on immediate basis

[Cyrus-Shepard]

Howdy,

A couple of questions:

1. Are there certain pages that aren't getting updated, or is it your entire site?
2. How often are changes in the pages reflected in Google's cache?

Is it a case where Google simply displays old/outdated information all the time? Finally, have you done a "Fetch and Render" check in Google Webmaster Tools?

[Modi]

@ Cyrus

Certain pages

Earlier it was less than 4 hrs - but now its taking around 24 hrs - basis the data been updated in Search engine result just found today - i thought it was not getting updated at all

Fetch & render - no issues, Its submitting. No errors in GWT

Tested speed test - though no noticeable improvement in loading time - but no unnessary page size or load time been increased too

I was wondering - can it be a temporary phenomena - where crawl speed is slow and later on will come back to normal. Its less than 72 hrs when server been migrated

Google Search Console Crawl Stats is last updated for 16th June - so unable to figure it out from there. No errors in webmaster

[Cyrus-Shepard]

The good news is, this actually sounds pretty normal. 24 hours to reflect changes in content is better than many sites. I can't account for why it dropped from 4 to 24, but I'd say this is still in the range of "good"

[ThompsonPaul]

The one thing you haven't mentioned, which is likely to be most critical for this issue, is your XML sitemap. I couldn't find it at any of the standard URLs (/sitemap.xml and /sitemap_index.xml both lead to generic 404 pages). Also, there's no directive to the sitemap in your robots.txt.

Given that the sitemap.xml is the clearest and fastest way for you to help Google to discover new content, I'd strongly recommend you get a clean, dynamically updated sitemap.xml implemented for the site, submit it through both Google and Bing webmaster tools, and place the proper pointer to it in your robots.txt file.

Once it's been submitted to the webmaster tools, you'll be able to see exactly how frequently its being discovered/crawled.

Hope that helps?

Paul

[Modi]

Thanks all for Inputs

I searched Google and found this note from Google which may happen post server migration

https://support.google.com/webmasters/answer/6033412?hl=en&ref_topic=6033383

A note about Googlebot’s crawl rate

It’s normal to see a temporary drop in Googlebot’s crawl rate immediately after the launch, followed by a steady increase over the next few weeks, potentially to rates that may be higher than from before the move.

This fluctuation occurs because we determine crawl rate for a site based on many signals, and these signals change when your hosting changes. As long as Googlebot does not encounter any serious problems or slowdowns when accessing your new serving infrastructure, it will try to crawl your site as fast as necessary and possible.

Add on Thompson Paul - Appreciate - yes its a good suggestion, will see to include sitemap