4 ms·
> Chomsky derided researchers in machine learning who use purely statistical methods to produce behavior that mimics something in the world, but who don't try t
by 6ren 16y ago
> Chomsky derided researchers in machine learning who use purely statistical methods to produce behavior that mimics something in the world, but who don't try to understand the meaning of that behavior
[He's not deriding these, but] statistical methods can be used to infer models: you have a series of models, and you measure how well each one models the data, and you include a measure of the complexity of the model (e.g. the choices (information) needed to specify that model). The model requiring the least information wins (related to Occam's Razor).
- woodson 16y agoYeah, but a relevant measure of complexity is not that easy to come by. And by what measure do you judge which is more 'valid'? For example, AIC, BIC etc. may have a strong effect on which model 'wins'. You are right of course, but I'm wary of making claims based on inferences from model comparisons without stating such limitations.
- 6ren 16y agoYes, measure of model complexity is a problem. A solution is to use a program for a Turing machine as the model. The length of the program represents the complexity (which measures the choices that form the program). There's different formulations of a turing machine (different languages for programming one), but since any turing machine can be simulated by any other turing machine with a program, the length of that program becomes the error (that represents the difference in complexity introduced by your selection of some specific Turing machine). If you are comparing models using the same turing machine, then this constant difference doesn't matter. a-b = (a+x) - (b+x) One practical solution is that more data swamps minor differences between the specification language used to describe models. More data is better.