3 ms·
That's... just not true. We know these models capture aspects of the training dataset, otherwise training wouldn't be a thing. That's the entire point. We al
by BaseballPhysics 4y ago
That's... just not true.
We know these models capture aspects of the training dataset, otherwise training wouldn't be a thing. That's the entire point.
We also know these models create a very compact representation of the data they derive.
So it is perfectly reasonable to think of these things as a compressed representation. Is it lossy? Yes, of course. But so what?
- tomrod 4y agoA valid model is f(all inputs imaginable)=2. No weights involved. You can apply as much training as you like, but its still just the value "2" as an output. There is no representation of the data. ML models are a mix of two things: 1. Patterns defined by the individuals performing the model training. Commonly this is referred to as model architecture, but ultimately its the cobbling together of various items to create a way to get a signal from data in a certain way. (As a side note, thanks Statisticians and Computer Scientists for leading the way here!) 2. Model weights to minimize the uncertainty in those patterns. How that uncertainty is measured is part of (1).
- pixl97 4y ago>Is it lossy? Yes, of course. But so what? "But so what" is missing the entire point of intelligence. You as an intelligent creature only work because of this lossiness. Reality presents far too much information to process and instead brains, especially in higher lifeforms have evolved to filter out as much useless information as possible in the most energy efficient way, while not losing important context.
- Riverheart 4y agoRight there's over/under fitting. Train SD on the same data too many times it over produces it in results. Just because they tailor it to get more nebulous results doesn't mean the data doesn't exist in some form.