Sitemap generator

[nezona]

Hi There,

What is the best tool for XML Sitemap generation. I use one at the moment but it seems to be missing URL's out.

[Bryan_Loconto]

Hey Neil!

Check out this post on the topic. It should shed some light for ya!

Which one are you using currently that is having issues?

Cheers

http://moz.com/community/q/the-best-automated-xml-sitemap-generator

[nezona]

xml-sitemaps.com my client has advised it is missing a lot of pages.

[DonnaDuncan]

I use screaming frog. I mention it because I don't see it being discussed in the helpful post Bryan referenced and because screaming frog has many other uses for SEO, not just sitemap generation. There is a free and paid version of the tool. The free version will allow you to scan (and generate a sitemap for) up to 500 pages. The paid version is a 12 month license costing 99 UK dollars per year.

[TimHolmes]

I have previously used Xenu Link Sleuth which unlike Screaming Frog will allow a full listing without a subscription, it will also generate an XML sitemap too.

So far when used it has seemed to be pretty accurate at pulling back everything I have wanted.

Here is an old post on its uses from Search Engine Journal.

Hope it helps.

Cheers

Tim

[Devanur-Rafi]

Hi Neil,

My personal favorite would be ScreamingFrog SEO Spider as recommended by Donna. Its highly recommended to go in for a yearly subscription and it worth every penny. It gives you a wealth of information that can be handy for a quick SEO audit. Used to use Gsite crawler earlier about till 3 years back but have been using ScreamingFrog SEO Spider and it has never let me down. You have a ton of options like being able to select User agents, regex for including or excluding certain sections or to crawl only certain sections of your website, list mode for finding HTTP header status codes for thousands of URLs along with many other options in no time. I use it daily.

Best regards,

Devanur Rafi