Story time: I was playing with the kaggle lending club dataset and getting really high accuracy (high 90s) predicting default with an out of the box sklearn mo

Story time: I was playing with the kaggle lending club dataset and getting really high accuracy (high 90s) predicting default with an out of the box sklearn model. Just for fun I ran it through LIME and discovered that every single default was strongly predicted by the "recoveries" feature. I looked into the data dictionary (yeah, I should have done so first...) and discovered that this feature indicates the amount of debt recovered by collections agencies...