Welcome to the Q&A Forum

Links that have been discounted do indeed factor into penalties. In fact, they're probably the links you want to remove FIRST because these are sites/pages that Google has already flagged. You should absolutely remove them, especially if you're under penalty of some sort. Removing links is indeed a bit of a two-edged sword in that you often cut out some spam links that Google doesn't (yet) know about. That said, leaving the links in place is the poorer option in my view, as it prevents you from moving forward with a long-term strategy. If all of your links are manipulative, it might be better just to start a new site rather than cleaning up to return to 0.