5 ms·
Heisenbugs in AI are also interesting. As computers approach sentience, it will become progressively harder to explain their behavior. Why did you eat a bagel
by shawn 9y ago
Heisenbugs in AI are also interesting. As computers approach sentience, it will become progressively harder to explain their behavior.
Why did you eat a bagel instead of eggs this morning? Usually there isn't a scientific answer.
In the present, AI models are already so complicated that it seems hard to get reproducible diagnostic results, short of just saving every frame of data that the car's sensors pick up. And for video that seems rather prohibitive. Imagine collecting all sensor data for each of those 200 trips per day along the entire stretch of road.
- buvanshak 9y agohttps://en.wikipedia.org/wiki/Catastrophic_interference https://en.wikipedia.org/wiki/Catastrophic_interference >Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to completely and abruptly forget previously learned information upon learning new information.
- tensor 9y agoThese self driving systems are not likely learning in realtime. When the algorithms are updated, best practice is to validate their performance on a large regression set. So something like this should be caught during validation, assuming your test data is good.
- throwaway2048 9y agoNo set of test data can exist that covers every real world situation.
- smartician 9y agoAren't many AI methods also inherently probabilistic? That would make it by definition impossible to definitively explain any particular behavior.
- tytytytytytytyt 9y agoI don't think that means it's impossible. You can still log all inputs and sources of entropy.
- baddox 9y agoWhat do you mean by probabilistic? If you mean that the models takes input and outputs one or more labeled probabilities (e.g. “90% confidence the input photo is a dog, 10% confidence the input photo is a cat), then yes, I believe that many AI systems work this way. If you mean random, in the sense that the system may return different results given the exact same inputs, then in not sure if there are AI systems that work that way.
- joshuamorton 9y agoThere are. Monte Carlo methods is the keyword you're looking for. AlphaGo (Monte Carlo Tree Search) is an example of one such AI. Obviously you can set the RNG seed to be the same every time too, but even that only works if your system is wholly synchronous, which a car probably isn't. Note that I doubt Monte Carlo methods are common in the autonomous vehicle space.
- TaylorAlexander 9y agoWell I don’t work in autonomous cars but I did work on several autonomous robots, and “ Adaptive Monte Carlo Localization” is a common way to keep a ROS + LIDAR based robot localized. I wouldn’t be surprised if Monte Carlo techniques are useful for all manner of things related to ingesting sensor data on autonomous vehicles.
- chaboud 9y agoI'd expect Monte Carlo methods to be used in a number of cases that have deterministic time envelopes for evaluation. Randomized selection and evaluation can be incredibly effective. They also resist degenerate structured input vulnerabilities. I'm not in the automotive space, but I'd be surprised if there were a viable self-driving car team not using Monte Carlo methods somewhere in the vehicle stack.
- azinman2 9y agoId be willing to take a 1k bet that computers won’t approach sentience in our lifetime. Don’t believe the AI hype train, there hasn’t been a real breakthrough. Just more accurate versions of the same (dumb) classifiers.
- baddox 9y agoWhy do you think that humans are not just even more accurate versions of the same types of classifiers?
- neffy 9y agobecause we invented cars. And a few other things.
- dorgo 9y agoAmong these other things which we invented are algorithms which invented other algorithms to solve real world problems. There is nothing unuique or irreplicable about humans (save of culture, emotions, etc. which nobody wants to replicate).
- neffy 9y agoWhile it may come to pass that we do those things, so far all we've managed to replicate is the first 6 or so levels of the visual processing field. Methinks there is far further to go than we realise - especially with the current air of hubris around the entire endeavour.
- azinman2 9y agoBecause we integrate, learn, plan, abstract, create, have way more senses, complex memory, societal and cultural embeddings, and a whole host of things that make us human. A text box on hacker news is not the place for such a reply. Go take some cognitive science, neuroscience, anthropology, sociology, psychology, and philosophy classes and you’ll see why. “AI” as we know it is a joke, particularly as media has hyped it. The original guys (McCarthy, minskey, etc) at least understand the gigantic complexity of the undertaking in so far as we understand how the mind works at all, and they never reduced things purely to classifiers. That’s like 0.1% of the job. The real researchers take philosophy of mind seriously — technologists want a quick and cheap solution that cannot exist.
- joesmo 9y agoIf they can't figure out how to store that data they shouldn't be on the road. You're telling me it's a data storage / transmission problem that their best engineers can't solve, yet we're supposed to trust their machine learning system? Tons of companies are working on this and plenty seem to have figured out even bigger big data issues already.
- jobigoud 9y agoWe also see this in unexplainable flash crashes from high speed trading softwares gone bersek. The speed at which they handle data is so immense than going through the logs of a few seconds around the issue could take several years of human time.