Questions
-
How do I stop the Moz crawler from returning errors for another domain.
Hi Jason, First, just to check, is google indexing and crawling to demo.xxx.com? If it is then you will have duplicate content and you will need to do more than just stop moz crawling it. However, if it is just Moz crawling it... Navigate to the robots.txt within the root directory of the sub-domain (demo.xxx.com) and add the lines User-Agent: rogerbot Disallow: / That will stop good old roger from crawling over demo.xxx.com
Other Questions | | ATP0