Welcome to the Q&A Forum

It will not be wise to come up with a suggestion without looking deeply in to your website. There is a strong possibility that penalty hit your website because of your link profile and not your link structure... Please drop your URL and if the problem is in your link profile then all you have to do is to find out the bad links and kill them accordingly! As a result you will see your rankings will started to come again with the time... if the problem is on the content on usability site then we probably need to look in to changing the content and site structure of the website. At the moment all suggestions are quite blur.. I would appreciate if you drop your URL in order to get some solid information regarding your website.