3 ms·
The basic problem is that machine learning systems don't really understand what they're doing. So most of the time, they do the right thing, and when they do a
by Animats 4y ago
The basic problem is that machine learning systems don't really understand what they're doing. So most of the time, they do the right thing, and when they do a totally wrong thing, they don't notice. Works fine for advertising, not so much for self-driving.
Look at those systems that generate images or text from a prompt. Usually the results are good, and sometimes they are totally bogus.
As I point out occasionally, the big hole in AI is "common sense", defined as not screwing up big-time in the next 30 seconds. Until that gets solved, AI systems can't be trusted very far.
The control theory people are trying to fix this, so they can use ML to build control systems with safe behavior. The math is really tough. Way beyond me. See IEEE Transactions on Control Systems Technology to follow this. People are trying to make control theory math and ML math play together. Control theory usually has continuity - if it does the same right thing at 0.21 and 0.22, you can be confident it will do the same right thing at 0.215. ML systems do not have that property. Which is why we see those image recognition demos where some minor change in the background noise totally changes the result.
- drooby 4y agoCan’t we create a model that asks questions when the other model gets something wrong. The model will get very good at asking questions and then the other model will update accordingly..millions of people using the system will keep it continuously improving itself.
- pjbeam 4y agoWon't the watcher model then need a watcher? And the watcher watcher model too? Etc.
- ben_w 4y agoThat’s basically what a GAN is — they’re great, but not a panacea.
- version_five 4y agoDoesn't really work that way in that it still reduces to an overall system that doesn't understand what it's doing (someone else mentioned GANs, they are a great example of how unwittingly incorrect ML model output can be - go look at the out-takes from any image generation model) A better approach is good uncertainty quantification (this is not at all trivial) to quantify when the model is and isn't qualified to be making a prediction.
- ben_w 4y ago> The basic problem is that machine learning systems don't really understand what they're doing. So most of the time, they do the right thing, and when they do a totally wrong thing, they don't notice. Agreed, but how is the different from humans?