MOZ Crawl - Search engine blocked by robots .. but it isn't.

[webpred]

This post is deleted!

[Mobile_ID]

I am having the same issue here. It was working before we changed our platform. Google, Bing crawl my website w/o any problem, only Moz refused to crawl my pages. error says "605 : Home page banned by robots.txt, X-Robots-Tag: header, or tag." my meta name="robots" is INDEX, FOLLOW, nothing else. I need help too.

[ChiarynMiranda]

Hey Mike,

It looks like you submitted a support email on Thursday, and I followed up with you on Friday in that request. Since this seems to be a bug in our crawler, we will need to handle the issue in the support ticket so our engineers have access to the campaign information. They are investigating now and I will follow up with you in your support email once I have an update from them.

Thanks,

Chiaryn

[ChiarynMiranda]

Hey There,

Can you please email your campaign information to help@moz.com so we can investigate the issue further? I'm afraid we aren't able to investigate bugs through our Q&A forum.

I look forward to hearing from you there soon.

Chiaryn

[conexion33]

Hello! could you fixed that issue,  I´m having the same problem

605 : Home page banned by robots.txt, X-Robots-Tag: header, or tag.

Thanks a lot,

David

[ChiarynMiranda]

Hey David,

I took a look at your account and it looks like the campaign you are receiving this error on is actually blocking all crawlers from accessing the site. Without providing any identifying information, here is a screenshot of the the robots.txt file which bans our crawler, as well as all other crawlers from the site: http://screencast.com/t/AD2JEo0ZXI

In order to allow crawlers to to access the site, you would need to either remove the / after Disallow or change Disallow to Allow.

If you specifically want to allow our crawler, you can insert the following directive above the current directive that is in the file:

User-agent: rogerbot
Disallow:

I hope this information helps. Please let me know if you have any other questions.

Chiaryn

[ozonetel]

I am also facing same issue, all the pages are getting Blocked by X-robots error. Can anyone tell me what is this ? and how to solve the issue?

[ChiarynMiranda]

Hey Murthy, without your site details, I'm not able to investigate what may be causing our crawler to be blocked from your site. Can you please email your campaign information to help@moz.com so we can investigate the issue further? Again, we can't really investigate this type of issue through the Q&A forum.