Can hosting blog posts with keyword anchor text on outbound links cause a penalty?

[RedNovaLabs91]

My site received a Google penalty for having inbound links from blog posts with over-optimized ("spammy") anchor text.  I spent months getting these links removed.

Yesterday - I received a link deletion request from a site that my site had linked out to (three links via keyword anchor text relevant to their company) in a blog post.

The "unnatural link" penalty still hasn't been removed from my site.

My question is:  Does the penalty work both ways?  For having inbound "unnatural" links ... AND for having outbound "unnatural" links?

[MarieHaynes]

If you received an inbound unnatural links penalty then the links going OUT from your blog will not matter.  It's possible for you to also get an outbound unnatural link penalty as well but you would receive notice of that just like you did with the inbound penalty.