Moz Crawler not working on beta site
-
Hello Mozzers,
We are launching a new site and before we do we wanted to run a crawl but unfortunately no diagnosis came back. The beta site is not password protected. It does have noindex,nofollow but it shouldn't matter. So I am not sure why nothing has been detected?
Thanks!
-
If it does have a no index no follow, then the crawler acted as expected, right? It didn't index the pages or crawl them.
-
I am confused because nofollow and noindex have always been detectable in the past with live sites. Is this new?
How am I suppose to go around this since I don't want to launch a beta site with tons of errors which could hurt the SEO?
-
Hi everyone!
I wanted to follow up with everyone who experienced the same issue. Unfortunately the moz crawler doesn't crawl any noindex,nofollow which proves its limits. There is another tool called screamingfrog that will allow you to crawl all the noindex, nofollow pages. This tool is literally saving me as I was able to detect many errors on the Beta Site. Without it we would have launched the new site and our SEO would have taken a major hit.
Thanks!