A couple of years ago, my first agency experienced this issue. There were a few queries I kept a sharp eye on, and the site disappeared for most of the day. The site couldn't even be found for the company name. Later in the day, the site returned to it's previous placement.
That happened a few months after noticing a couple clients received referral traffic from banyan.google.com, via Google Analytics. Basically, The Eye of Sauron was watching the properties. This didn't appear to bother the higher-ups, but it bothered me.
I had advised that they stop doing what they were doing, but they could see no ill end to it. They went on to survive the first Penguin. Penguin 2 proceeded to knock them out of the running. After I left, there was no recovery. They were happy to continue on their way.
Your site may return, but it's unlikely. Are the people that say it will return in a few days the very same that put it in the situation?