Editing A Sitemap

[94501]

Would there be any positive effect from editing a site map down to a more curated list of pages that perform, or that we hope they begin to perform, in organic search?

A site I work with has a sitemap with about 20,000 pages that is automatically created out of a Drupal plugin.

Of those pages, only about 10% really produce out of search. There are old sections of the site that are thin, obsolete, discontinued and/or noindexed that are still on the sitemap.

For instance, would it focus Google's crawl budget more efficiently or have some other effect?

Your thoughts? Thanks! Best... Darcy

[DmitriiK]

hi there.

Of those pages, only about 10% really produce out of search. There are old sections of the site that are thin, obsolete, discontinued and/or noindexed that are still on the sitemap.

Have you considered removing those pages/sections, rather than altering the sitemap? It would make more sense I think.

[ThomasHarvey]

"There are old sections of the site that are thin, obsolete, discontinued and/or noindexed that are still on the sitemap."

Why not remove these from the site?

I personally believe that it'll have a positive impact, as you're submitting this sitemap to Google, you're giving it a way of going through your whole site, so why would you give it low quality pages. You want to provide Google (and your users) the best possible experience, so if you've got out of date pages, update them or if they're not relevant delete them, a user who lands on this page anyway would just bounce because it's not relevant anymore.

If these out of date pages can't be found by crawling, then 100% it's best to craft your sitemap to show the best pages.

[Andrew_Birkitt]

Hi Darcy

I don't know about scaling the sitemap down but you could make use of an area of the sitemap to optimise and make it a crawl more efficient.

The area in question is the Priority area that basically tells the search engines which pages on your site are the most important. The theory is that pages with a higher priority (say 100%) are more likely to get indexed by the search engines than pages with a lower priority of say (10%), although not everyone in the industry agrees.

[94501]

Dmitrii & Thomas,

Thanks for your thoughts.

Removal would be one way to go. I note with some interest this post:

https://www.seroundtable.com/google-block-thin-content-use-noindex-over-404s-21011.html

According to that, removal would be the third thing after making it better and noindexing.

With thousands of pages, making it better is not really an option.

Best... Darcy

[ThomasHarvey]

Darcy,

Whilst noindex would be a good solution, if the page has no benefit why would you noindex instead of deleting it?

[DmitriiK]

You misunderstand the meaning of that article.

"...that when you do block thin or bad content, Google prefers when you use the noindex over 404ing the page..."

They are talking about the walk around the problem of blocking pages INSTEAD of removing them.

So, if for whatever reason you don't want to delete a page and just put a 404 status on it, it's worse than putting noindex on it. Basically, what they're saying is:

• if you have thin content, DELETE it;
• if for whatever reason you don't want to delete it, put NOINDEX on it.

P.S. My suggestion still stays the same. Delete all bad content and, if you really want, put 410 gone status for that deleted content for Google to understand immediately that those pages are deleted forever, not inaccessible by mistake or something.

Hope this makes sense

.

[94501]

Thomas & Dmitrii,

Thanks for the message. With all do respect, I really read the above linked post differently than Google saying "just delete it."

Also, I don't see how deleting it preserves whatever link juice those pages had, as opposed to a "noindex, follow" and taking them out of the sitemap.

Finally, I don't necessarily equate all of Google's suggestions as synonymous with a "for best effect in search." I assume their suggestions mean, "it's best for Google if you..."

Thanks, again!

Best... Darcy

[94501]

Thanks for the suggestion, Andrew.

With setting priority or not in a sitemap, do you think a smaller, more focused, sitemap is generally an advantage?

Thanks... Darcy

[94501]

Hi Thomas,

Thanks for the message.

To answer your question, part of the reason is link juice via a noindex/follow and then there are some pages that serve a very very narrow content purpose, but have absolutely no life in search.

All things being equal, do you think a smaller, more focused, sitemap is generally an advantage? In the extreme and on other sites I've seen sitemaps with noindexed pages on them.

Thanks... Darcy

[DmitriiK]

- I really read the above linked post differently than Google saying "just delete it."

Well, here is a video from Matt Cutts about thin content. In this particular video he's talking about websites, which already took hit for thin content, but in your case it's the same, since you're trying to prevent it
https://www.youtube.com/watch?v=w3-obcXkyA4&t=322

So, there are two options he is talking about: delete or make it better. From your previous responses I understand that making it better is not an option, so there is only one option left

As for link juice thorough those pages. If those pages have good amount of links, traffic and are quite popular on your website, then surely DON'T delete them, but rather make them better. However, I understood that those pages are not popular or have much traffic, so, option two

[94501]

Hi Dmitrii,

Always fun to watch Matt's Greatest Hits, in this example the value of making things better.

I guess the make better or delete seems super black and white to me.

Economically, who is able to make thousands of pages dramatically better with compelling original content? So, instead, the only other option is apparently radical elective surgery and massive amputation? I guess I'd choose the chemo first and don't really see what the downside is for noindex/follow and exclude from the sitemap.

Anyway, thanks again! Best... Darcy

[Andrew_Birkitt]

Hi Darcy

Looking at what has been mentioned previously I would agree with the train of thought that a more focussed sitemap would generally be advantageous.

Andrew