There is a similar thread here: http://moz.com/community/q/what-if-my-site-isn-t-ready-for-mobile-armageddon-by-april-21st
If you have a lot of mobile visitors coming in via Google, I think you may expect that this traffic will disappear (unless your competitors aren't very mobile friendly as well).
Nobody really knows what the impact is going to be, most discussions seem to agree that the impact will only be for mobile searches, but again, it's difficult to predict.
It also seems that it's defined on page - not site level, so you could try to provide mobile versions for your most important landing pages (dedicated or responsive). Your lay-out seems pretty straight forward, so I guess it shouldn't be too difficult to adapt it Remember that it doesn't have to perfect - it just needs to pass the "mobile friendly" test - https://www.google.com/webmasters/tools/mobile-friendly/ - you can always improve later on.
Hope this helps
Dirk