Will robots.txt override a Firewall for Rogerbot?

[HashtagHustler]

Hey everybody.

Our server guy, who is sorta difficult, has put these ridiculous security measures in place which lock people out of our website all the time. Basically if I ping the website too many times I get locked out, and that's just on my own, doing general research.

Regardless, all of our audits are coming back with 5xx errors and I asked if we could add rogerbot to the robots.txt. He seems to be resistant to the idea and just wants to adjust the settings to his firewall...

Does anybody know if putting that in the Robots.txt will override his firewall/ping defense he has put in place? I personally think what he has done is WAY too overkill, but that is besides the point.

Thanks everybody.

[ThompsonPaul]

Unfortunately, no. The security he has in place will block the crawler access before it ever gets a chance to see the robots file.

If you have a dev who's making business-limiting decisions,  you have a major problem and need to address that first.

Hope that helps?

Paul

[HashtagHustler]

Yes it does thank you!

When I asked out dev (who also hosts our domains) to adjust the settings for the rogerbot he said

"3 pages per second is basically me undoing the portflood setting completely, thus rending the site very insecure to brute force attempts, which would inevitably drive the server load very high in anywhere from 3-24 hours."

I am glad that he is concerned about the security of our website. At the same time, I find it hard to believe we need anything near this intense. We do not have any online store, we do not collect credit card data or anything like that.

It seems overkill...

[HashtagHustler]

So I spoke with our host. Basically he has been adjusting the port flood settings because of a DDoS attack we had roughly 9 months ago.

We have roughly 1000 domains on the same server, all with wordpress. I went through and changed the nameserver on around 800 of them to bring us down. In the long run, I want to bring us to 1 website. There is no reason for us to have 200, or 5 for that matter. They are redundant websites that were build simply to bolster our main website by blackhat tactics.

Our host stated that the only way to keep things kosher would be to switch all 1,000 domains to a new server every 2 years because once the "hackers" find out that there is a cluster of 1,000 domains in the same place, they will blast it.

Anyway, I'm working on cutting the domains in the safest way possible, and switching servers as soon as possible!