2 ms·
One interesting correlation with what you’re saying: in AI programming, there is the concept of a “loss function”, which is basically just a number that tells y
by sillysaurusx 7y ago
One interesting correlation with what you’re saying: in AI programming, there is the concept of a “loss function”, which is basically just a number that tells you how well your model is doing at a given task. For example, how well it’s predicting the next word in a sentence. The lower the loss, the better the quality.
Or that’s what everyone believes. Empirically that doesn’t seem true. Just because your loss goes down, quality doesn’t necessarily go up. It’s a statistical correlation, but it’s only a loose correlation at best.
So it’s kind of doubly related here: not only is the knowledge a statistical summary, which hides the details of what’s happening, but we also don’t know a better way to do it. No one knows how to measure quality other than to show some samples to a bunch of humans, or to use the incorrect loss function.
I suppose the point is that it’s very easy to accidentally hide such complexities using math. No one intentionally sat down and said “I’m going to come up with a very imprecise loss function that gives a misleading view of how well AI models are doing.” It just happened by default. And it seems like that might be related to your point of economists ignoring limitations, for example. The limitations are inherent in the statistical summary.