4 ms·
When you mention The Objective Information Criterion. I assume you mean something that takes into account the complexity of the model. A model with more paramet
by skyde 3y ago
When you mention The Objective Information Criterion.
I assume you mean something that takes into account the complexity of the model. A model with more parameters might fit the data better but could also be at risk of overfitting.
But isn’t there already a blackbox measure of overfiting one that focus on how well the model generalizes to new, unseen data, rather than on the model’s complexity. Like Cross-Validation, hold-out validation or bootstrap method.
- jabowery 3y agoThat's what all "information criteria for model selection" are about. The difference is that Algorithmic Information is the only such information criterion that has been proven (by Solomonoff) optimal under the assumptions of natural science.