Moz was unable to crawl your site on Sep 18, 2018

[khurrum]

You tell me how do i fix that? I have the robots.txt file in my domain. https://www.beencrypted.com/robots.txt

Thanks

[Roman-Delcarmen]

Ok, I made a quick test of your robot.txt file and looks fine,
https://www.beencrypted.com/robots.txt

Then I made a test https://httpstatus.io/ to check the status code
of your robot.txt file and show me 200 status code (So it's fine)

Also, you need to make sure that your robot.txt file is accessible for the Rogerbot (Moz crawler)
This day the hosting providers have become very strict with third-party crawlers
This includes Moz, Majestic SEO, Semrush and Ahrefs.

There have many other members of this community with similar issues recently

Here you can find all the possible sources of the problem and recommended solutions
https://moz.com/help/guides/moz-pro-overview/site-crawl/unable-to-crawl

If you're still having trouble I will suggest you write to Moz support team help@moz.com. Hope this info helps you

Regards

[meghanpahinui]

Hi there!

Thanks so much for the question! I'm sorry to hear you're having trouble getting the crawl started for your site!

In addition to everything that Roman addressed, I would also advise checking the scope of your Campaign settings. If your Campaign is set to crawl all sites within the domain of beencrypted.com it means that our crawler is attempting to access the robots file at http://beencrypted.com/robots.txt which doesn't resolve and errors out. I was able to confirm this in a browser as well as with the http status checker here.

You may need to speak with your web developer about the best way to fix this issue as it appears to be a problem with the redirect in place.

Another option would be to set up a Campaign which is restricted to the www subdomain of your site. To do so, you would need to click "advanced" during Campaign setup and make sure the box is checked. I have a guide here  to help get you started with that process, as well! This way you can get your crawl started.

I hope this helps to point you in the right direction! If you're still having trouble after looking into these factors, please do send us an email at Help@moz.com so we can investigate further.

[Netvilox]

Hi there!

Thanks so much for the question! I'm sorry to hear you're having trouble getting the crawl started for your site!

In addition to everything that Roman addressed, I would also advise checking the scope of your  https://9jahob.com/.