Issue with a particular command of robots.txt! Help needed!
-
Hi all,I'm facing a strange issue with a website: www.foodmood.itI've written this robots.txt: www.foodmood.it/robots.txt in which I'm using all commands I've tested and used on other websites.The problem is that this website is not indexing correctly, and my fear is that I've written the robots in a wrong way.In this forum I've been taught that this command Disallow: /*? should be substituted with Disallow: /? in order to be sure the crawler is not indexing all the URLs containing a question mark.My question is: do you think this command modified in the above way can cause a problem to my website?I repeat that I've used it on several sites and this is the first time I'm having this kind of problems.Thanks in advance for your help.A.
-
I would in future tend to trust sources closer to being official rather than someone on a forum. Remove the extra * so its just Disallow: /*?
You can find more info here - https://support.google.com/webmasters/answer/156449
You can also test your robots in your webmaster tools so don't forget to give that a go if in doubt.
Hope that helps.