How to resolve - Googlebot found an extremely high number of URLs

[Myntra]

Hi,

We got this message from Google Webmaster “Googlebot found an extremely high number of URLs on your site”. The sample URLs provided by Google are all either noindex or have a canonical.

• http://www.myntra.com/nike-stylish-show-caps-sweaters
• http://www.myntra.com/backpacks/f-gear/f-gear-unisex-black-&-purple-calvin-backpack/162453/buy?src=tn&nav_id=541
• http://www.myntra.com/kurtas/alma/alma-women-blue-floral-printed-kurta/85178/buy?nav_id=625

Also we have specified the parameters on these URLs as representative URL in Google Webmaster - URL parameters.

Your comments on how to resolve this issue will be appreciated.

Thank You

Kaushal Thakkar

[David-Kley]

"we have specified the parameters on these URLs as representative URL in Google Webmaster - URL parameters."

How long ago was this done? Since there are so many URL's, it may take a while for them to recrawl and index the representative URL's per your request.

[Myntra]

Thank you David, Its been more than 10 months since these parameters have been specified in webmaster. This and other activities like noindex and canonicals helped us to reduce the indexed URL count from 32 million to 1.2 million. As the url index reduced this warning from google stopped for 4 months. However we started receiving this message again from february 2014.

Thanks

Kaushal

[Paddy_Moogan]

Hi Kaushal,

Thanks for the question.

There are a few ways to deal with this problem which are recommended by Google here. In summary, you can:

• Use parameter handling as you have done
• Add the nofollow attribute to problematic URLs
• Block problematic URLs in robots.txt

There is also a thread in the Google webmaster forums which may be useful to you:

https://productforums.google.com/forum/?hl=en#!category-topic/webmasters/crawling-indexing--ranking/CCORJBI-mEg

Overall, it comes down to having a good site architecture and cutting down / removing / blocking URLs that you don't care about from a search perspective.

I hope that helps a bit!

Paddy