Site Maps

[Liammcmullen]

I have provided a site maps for google but although it craws my site www.irishnews.com at 6:45AM the details in the site map are not seen on google for a few days  -  any ideas how to get this feature working better would be great.

example

<url><loc>http://www.irishnews.com/news.aspx?storyId=1126126</loc>
<priority>1</priority>
<lastmod>2012-01-23</lastmod>
<changefreq>never</changefreq></url>

thanks

[mrdavidingram]

Hi Liam,

There is always a natural delay between crawling and indexing, and it's rarely instantaneous. Although I can see why you'd want a news site to be getting indexed pretty quickly.

The one thing that stands out is from the example is the <changefreq>tag, which you've got set on 'never'. This is essentially for archived pages, and tells crawlers that it's low importance (even though you've given it high priority). Even if you're not intending to change the article, I'd still recommend giving it a change frequency of 1 month so you're inviting the crawlers to come and check it more often. Saying that, this doesn't mean that if you set the frequency to hourly that the crawlers would come back every hour, as they'd soon figure out that nothings changing.</changefreq>

Really I'd have your home and catergory pages on daily, the articles on monthly, and the static pages on monthly or yearly.

In terms of getting them indexed quicker after the crawl, this is just a case of establishing trust and importance from the search engines. They need to know that you have news content that requires a quicker indexed. You can gain this trust by providing regular, high quality pages. When their crawlers pick up that there are new pages going live daily you will see the index get quicker.

Depending what CMS you are using, it is also worth getting a script or plugin that pings the search engines with an updated site map every time a new page or post is added. I've done this through Wordpress and articles get indexed within 4 hours. Again though, it didn't start this way and it only got this quick once I was putting up 4 posts a week like clockwork.

Good luck.

Cheers

David