Moz crawl showing up ?s=keyword pages as errors
-
Hi all,
Hoping someone can she some light on a fix with ref to wordpress and the search function it uses as Moz is craling some pages which reference the search
Errors showing up are duplicate pages, descriptions and titles.
The search function is not important on this site and I have tried to use a plugin which disables the search page which it does but these errors still show up. Can anyone assist as this is the final piece of the puzzle and then we're down to 0 issues on the site.
-
Different ways to do this - best way would be to eliminate the search from your template in the editor files. If you want Moz to stop crawling your internal search pages add this to your robots.txt file:
User-agent: RogerBot
Disallow: /?s=*?
This should block Moz's crawler from any page beginning with _/?s= _after the root domain. Do a regex test just to double check your internal search URL and that piece of regex.
-
Hi there,
I am not sure which fix will stop Moz from flagging this, but just in case it is also being picked up by Google, are you able to serve the canonical tag on domain.com/?s=keyword and any search query pages, pointing to a generic page?
You can remove parameters from Google in Webmaster Tools as well: https://support.google.com/webmasters/answer/1235687?hl=en
This ensures that Google does not crawl pages with the parameter you specify, meaning they won't be an issue in its index.
-
I have implemented the robots method for the moment and I will look at the exclusion of parameters too. thanks , i will now wait the next crawl from Moz .