Welcome to the Q&A Forum

Yes, it's in the robot.txt, that's the problem. Someone had to physically put it in there, but I've no idea why they would.

Hi

I have a website with thosands of products. On the category pages, all the products are linked to with the code “?cgid” in the URL. But “?cgid” is also blocked in the robots.txt file for some reason. So I'm thinking it's stopping all my products getting crawled by Google.

Am I right here? Is there any reason why a website would want to limit so many URL's? I'm only here a week and the sites getting great traffic, so don't want to go breaking it!!!

Thanks