Client accidently blocked entire site with robots.txt for a week

[pixelpointpress]

Our client was having a design firm do some website development work for them. The work was done on a staging server that was blocked with a robots.txt to prevent duplicate content issues.

Unfortunately, when the design firm made the changes live, they also moved over the robots.txt file, which blocked the good, live site from search for a full week. We saw the error (!) as soon as the latest crawl report came in.

The error has been corrected, but...

Does anyone have any experience with a snafu like this? Any idea how long it will take for the damage to be reversed and the site to get back in the good graces of the search engines? Are there any steps we should take in the meantime that would help to rectify the situation more quickly?

Thanks for all of your help.

[rsanchez]

Hi, have you tried logging in to Google Webmaster tools and fetching the URL as googlebot?  This helped me recently with a couple of sites that I had blocked with robots.txt.  They were up-to-date in SERP's within 2 days.

[EGOL]

I can't say anything about robots.txt

.... but one of my competitors tossed up a new design with nofollow, noindex tags on every page and their site immediately tanked out of Google.

... it took them a couple weeks to figure it out but once they yanked that line of code they were back at topSERPs within 48 hours.

... this was a relatively strong site and I would expect that type of site recovers faster than a PR2 site with little connectivity.

[KeriMorgret]

Here's a YouMoz post that was promoted to the main blog about what someone else did in this situation that may help.

http://www.seomoz.org/blog/accidental-noindexation-recovery-strategy-amp-results

A couple of preventative steps would have been to make the robots.txt file on the live site read-only so it couldn't have been as easily overwritten, and to use a free service like Pole Position's Code Monitor (https://polepositionweb.com/roi/codemonitor/index.php) to monitor the contents of your robots.txt file once a day and email you if there are changes. I'd also monitor your dev robots.txt, just to make sure the live site robots.txt doesn't get copied over to dev one day and your dev site gets indexed (I've had that happen!).