5 ms·
Generalized Information: A Method for Judging Machine Learning Models
- johnnyb_61820 7y agoSolomonoff induction has long been known to be important in machine learning, where the shorter of two equivalent models is considered "better". This paper extends this in three ways. First, it shows that the data itself can be considered a model of itself. Therefore, an absolute bound for being able to decide that you have a good model from the data is having a model that is smaller than the data. Second, the paper provides a method to quantitatively trade-off accuracy and model size. Third, this methodology is implementation-agnostic, and can in theory be applied to any type of machine learning model.