Add selective URLs to an XML Sitemap

[AB_Newbie]

Hi!

Our website has a very large no of pages. I am looking to create an XML Sitemap that contains only the most important pages (category pages etc). However, on crawling the website in a tool like Xenu (the others have a 500 page limit), I am unable to control which pages get added to the XML Sitemap, and which ones get excluded.

Essentially, I only want pages that are upto 4 clicks away from my homepage to show up in the XML Sitemap.

How should I create an XML sitemap, and at the same time control which pages of my site I add to it (category pages), and which ones I remove (product pages etc).

Thanks in advance!

Apurv

[Schwaab]

I would recommend purchasing Screaming Frog.  You can crawl the site and sort the URLs by level.  Remove the URLs that are too deep from the crawl and export to XML sitemap.  Screaming Frog is definitely worth the price to unlock all of its features and have an unlimited crawl limit.

http://www.screamingfrog.co.uk/seo-spider/

[Travis-W]

If you have a list of all the URLs on your site, it is easy to create a sitemap using excel.  I have a template that I use and I can crank out a 50k URL sitemap in 5 minutes.

[AB_Newbie]

Hi Travis

That sounds like a smart way to go about this. Could you please guide me regarding how to add parameters like lastmod, priority, changefreq etc in the XML sitemap, using the URLs that I have in the Excel sheet.

Thanks!

[AB_Newbie]

Thanks Schwaab!

[Travis-W]

Hey Intermediate,

Here's my setup - image - http://screencast.com/t/qThC401hQVUp Be careful of the line breaks if you want your sitemap to be pretty (I'm not sure if it also works if everything is on a single line).

Column A:

Column B:
URL

Column

<lastmod>2013-08-27</lastmod>

Column
<changefreq>always</changefreq>

Column E:
<priority>1</priority>

Column F:
=CONCATENATE(A2,B2,C2,D2,E2)

You will need to add this as first 2 lines in your sitemap:

and add to the end, but you should be good to go!
I Hope that helps!

[AB_Newbie]

Thanks a lot for sharing Travis. This is really helpful!

Appreciate your help here.