4 ms·
Missing data imputation (2021)
- aquafox 4y agoInmputing missing, in my opinion, is never a good choice. The main reason is that any subsequent method does not know which data were imputed and which not, i.e. it assumes the same error distribution. Using methods that can deal with missing data, preferably by ignoring them, e.g. by removing the corresponding terms from the likelihood, should always be preferred. Adapting methods to deal with missing data is often easily possible but not done because it requires to treat many edge cases.
- unlikelymordant 4y agoI agree with you about imputation being suboptimal, but there really arent that many classifiers that can natively deal with missing features. Bayesian methods are the best, where you can marginalize over any missing variables, and a few tree based methods seem to have capacity for handling missing. But Things like logistic regression require imputation. And neural nets may be able to get away with some missing data if you have lots of exmamples of missing data in the training set, but its not really native. A lot of classifier libraries just can't deal with missing stuff, so poor imputation is unfortunately necessary. The thing with bayesian classifiers is that you generally need to understand the problem domain a lot better to get good results out of them than e.g. neural nets.