Is the If-Modified-Since HTTP Header still relevant?

[annieplaskett]

I'm relatively new to the technical side of SEO and have been trying to brush up my skills by going through Google's online Web-master Academy, which suggests that you need a If-Modified-Since HTTP Header tag on your site.  I checked and apparently our web server doesn't support this.

I've been told by a good colleague that the If-Modified-Since tag is no longer relevant as the spiders will frequently revisit a site as long as you regularly update and refresh the content (which we do).

However our site doesn't seem to of been reindexed for a while as the cached version's are still showing the pages from over a month ago.

So two question really - is the If-Modified-Since HTTP Header still relevant and should I make sure this is included?

And is there anything else I should be doing to make sure the spiders crawl our pages? (apart from keeping them nice, fresh and useful)

[SimonCullum]

Hi Annie

I'm surprised there hasn't been lots of answers to your question.

Check-out this video here on SEOmoz entitled  "Whiteboard Interview - Google's Matt Cutts on Redirects, Trust + More" featuring Matt Cutts being asked some questions by Rand.  It opens with a partial answer to your first question:

"These days we use it a little less" (2 years ago)  ~  basically means that in locations such as the US, most of Europe, Japan... & so on, where Bandwidth is rarely an issue anymore, 'If-Modified-Since' isn't taken notice of, it's not worth including anymore.

In say developing countries where bandwidth is sometimes still on the low side, it may still be used, hence why a sweeping 'it doesn't matter anymore' statement wasn't given.

**Your second question:  **

• Content, fresh unique value-adding content that is, that's engaging and shareable, is always a positive aspect to work on, which in turn can lead to some awesome new links. This encourages the bots to visit more regularly.
• Ensuring that your site doesn't have any technical issues (say causing significant downtime).
• Ensuring that Robots.txt isn't wrongly disallowing any pages from being crawled.
• Keeping an eye on Google Webmaster Tools (& Bing Webmaster Tools) for any messages or errors.
• You can alter the crawl rate in GWT, though is usually best to leave it on the default auto setting.

Hope that helps,

Simon

[jmueller]

If the webserver does not support (or the admin does not want to enable) this feature you could always have your frontend-templates have a small string wich holds the date/time when the page was last updated. Something along the lines "last updated on: ...." at the bottom or top of the content area. It's also an useful bit of information for users.