Removing massive number of no index follow page that are not crawled

[Digitics]

Hi,

We have stackable filters on some of our pages (ie: ?filter1=a&filter2=b&etc.). Those stacked filters pages are "noindex, follow". They were created in order to facilitate the indexation of the item listed in them.

After analysing the logs we know that the search engines do not crawl those stacked filter pages.

Does blocking those pages (by loading their link in AJAX for example) would help our crawl rate or not? In order words does removing links that are already not crawled help the crawl rate of the rest of our pages?

My assumption here is that SE see those links but discard them because those pages are too deep in our architecture and by removing them we would help SE focus on the rest of our page. We don't want to waste our efforts removing those links if there will be no impact.

Thanks

[OlegKorneitchouk]

noindex means that crawlers can still visit the page (using crawl budget). You would need to link to those pages using a nofollow tag + block via robots.txt to prevent crawlers from accessing them.

Overall, if those pages aren't being crawled currently, then they aren't affecting your crawl budget since they aren't being visited. However, if you build more authority to your website, your crawl budget will grow so crawlers might start visiting those pages again.

[MickEdwards]

Personally I don't agree with setting internal filter URLs to nofollow.  I set noindex as you have done and add the filter attributes to the Search Console > Crawl > URL Parameters.

For the option "Which URLs with this parameter should Googlebot crawl?" you can set "No URLs" (if the filters are uniform throughout the site).

"No URLs: Googlebot won't crawl any URLs containing this parameter. This is useful if your site uses many parameters to filter content. For example, telling Googlebot not to crawl URLs with less significant parameters such as pricefromand priceto (likehttp://www.examples.com/search?category=shoe&brand=nike&color=red&size=5&pricefrom=10&priceto=1000) can prevent the unnecessary crawling of content already available from a page without those parameters (likehttp://www.examples.com/search?category=shoe&brand=nike&color=red&size=5)"