Check this post out. In reference to a similar question.
Basically the TLDR is that unless it looks like blatant PBN/SPAM/Manipulation google doesn't seem to concerned. I have plenty of backlink that are 75% spam score and higher. Obviously I don't recommend looking for links that are super high spam score but I would put your effort into link acquisition as opposed to link cleanup.