4 ms·
To me the problem of AI is not how it works but how reliable it works. Do we have a way to test AI to prevent corner cases which could lead to catastrophic res
by croes 4y ago
To me the problem of AI is not how it works but how reliable it works.
Do we have a way to test AI to prevent corner cases which could lead to catastrophic results?
- CuriouslyC 4y agoTo be fair, this is a problem with people too. We use "interpersonal weirdness" as a proxy for "may produce catastrophic results" to some degree.
- ben_w 4y agoWe can't even do that with humans. Best we've got is qualifications, and punishments for breaking rules.
- dmichulke 4y agoWhat you're saying is (kinda) my response when people ask me about self-driving cars: How does the car deal with object X on the road? For X=piano, a stack of solar panels, a tank, an airplane, a pile of stones ... No one knows and since the size of the set of X is infinite, no one can appropriately train for it either. That's why we can't have self-driving cars without a general understanding of what objects are and how they move.
- pixelfarmer 4y agoYes and no. Yes, because I agree with what you said. No, because it assumes self-driving has to rely purely on normal vision/cameras, which is just garbage². Think LiDAR and other such means to detect obstacles, but things like that are (currently) pretty expensive, so there is the attempt to just do it with (more or less) normal cameras, which leads to what you said (and I fully agree with that). Aside of the fact that safety assumptions+validations towards cameras are complicated in itself already, not only the technology in all its aspects, but also dirty lenses and things like that.
- williamcotton 4y agoLiDAR can't tell you mass of an object as it is headed towards your vehicle. Imagine you're driving down the road and you see: 1.) A kid throw a rubber playground ball into the road and at your windshield. 2.) A kid heave a bowling ball into the road and at your windshield. How would you react in each case? How do you distinguish between the two? As a kid is heaving the bowling ball you can tell that it has a lot more mass by how the kid had to move his body to throw the ball. You learned that because you spent the first few years of your life experiencing the basic physics of reality. You've internalized the motion of the human body. LiDAR can also have a dirty lens!
- kortex 4y agoSame for both: brake. I don't want either hitting my windshield. The kickball may be bouncy, sure, but it may have gotten rocks stuck to it, which can create a point stress and scratch paint or chip the windshield. Humans can't intuit mass unless the object meets certain criteria, either. What if it's an opaque cardboard box in the road? Generally the strategy is the same: avoid it.
- williamcotton 4y agoIn certain circumstances with a playground ball you would have to choose between extensive damage to the body of your car by swerving out of the way or to just take the risk of a chipped windshield from a playground ball that somehow has rocks stuck to the outside. And have you seen ever seen a rubber playground ball with rocks stuck to the outside? Do you really hesitate to barefoot kick an unexamined playground ball that is bouncing in your direction? I mean, I literally did this yesterday. I was out barefoot in the neighborhood with my 1 year old and the neighbor kids were kicking a ball around and it bounced in my direction and I kicked it back to them. "Treat every object as the same" is not at all intelligent behavior for a man or a machine!
- AstralStorm 4y agoIn case of a car, the thrown object is the easy case. Now imagine you spot a person walking down the road on a pavement. Suddenly, the person turns towards the edge. What do you do? See, a human driver would look at the gait of the person from far away to evaluate for instance if they're sober, or if it's a child who might be expected to run in. Whether it is near a crossing or a potential crossing. Whether the person was walking or standing... Many other obvious and less obvious indicators. AI currently sees a moving blob of pixels in a shape of a person. No advanced inference. When interviewed in case of an accident, say because they got rear ended due to braking, a person can explain why they braked most of the time. AI now couldn't even say which features it weighted.
- sebzim4500 4y agoI don't see why self-driving cars should have to be perfect though. I would argue that rationally I should start using self-driving cars as soon as they are better at driving than I am. The fact there are edge cases that I handle better than the AI is not necessarily disqualifying, so long as the AI is so much better at the rest of the cases that on average the AI is safer. For example, the fact that I handle a piano that has been left in the middle of a freeway better than an AI might matter less than the fact that the AI responds 100ms faster when a car swerves into our path.
- notahacker 4y ago> I would argue that rationally I should start using self-driving cars as soon as they are better at driving than I am. How would you know? Of course the edge cases matter, since human drivers make minor driving mistakes on a too-regular basis but fatal ones very rarely. 100ms faster reaction time is very impressive up until the point the system interprets the 10,000th bridge support it passes as an offramp and attempts to exit via it, which makes it several orders of magnitude more deadly than humans as they start using it on that road...
- criddell 4y agoIn the US, more than 100 people are going to die in their car today. If hardware and software flaws kill people at a lower rate than the shitty drivers they replace, then that's a win.
- dmichulke 4y agoI agree with you. However, we can't really meaningfully quantify it a-priori. So we need a sufficiently big set of accidents and accident free rides to see when it's better. In addition we can't distinguish what driver would have been better or worse than the AI in a given situation. This opens the door for all sorts of lawsuits and we will end up with legal self-driving systems in almost ideal situations only. To note, this is as much an engineering problem as it is societal and legislative.
- criddell 4y ago> To note, this is as much an engineering problem as it is societal and legislative. Yeah, it definitely is. I think there are two things that should happen. In the short term, producers of self-driving cars (or systems) should be given immunity from lawsuits related to crashes provided that the NHTSA (or some other authority) can verify that the deaths per passenger mile in their cars does not exceed the rate for traditional cars. Once the majority of passenger miles are in self-driven cars, revoke the immunity. For the long term, software engineering should become a real engineering discipline and we require self-driving systems be signed off by licensed Professional Engineers. If those engineers ship software with bugs that lead to death or damage, those engineers can be held responsible, sued for malpractice, and have their license to practice revoked.
- notahacker 4y agoI'm not sure that the political 'solution' of giving special privileges to avoid immediate consequences from their actions to AV manufacturers and special responsibility for signoff in the longer term solves the engineering problem that they need each minor version update of their software to have driven tens of billions of real world miles before they remotely approach statistically significant evidence that it is less deadly than the last version, or indeed an average human. And this is where lack of general AI or tractability is an issue; knowing that the software typically makes fewer driving errors than the last version over a typical route gives us no confidence whatsoever it doesn't handle rare edge cases marginally worse leading to a couple of extra fatalities per billion miles (making it less safe than the average driver, many of whom suffer actual legal consequences for their erratic driving killing a person even if all their other driving is 'above average'...). Humans aren't bug-free or particularly tractable either, but at least we have enough of a mental model of how they understand driving to be confident that training them on a certain road sign found in urban areas won't make them more likely to stop on a freeway.
- pixelfarmer 4y agoI read neural network (NN) books in the 90ies, and they covered for example the problem with corner cases already. One thing I remember is a (comparatively simple) NN which created the past tense of an english verb, and there are exceptions like "go" and "went". It preserved that for a while, but further training brought it back to "goed", because NNs do generalize and you need tons of additional "storage" in a NN to keep such corner cases around. That also means if you need (near) 100% reliability you are in the wrong place when it comes to NN aka "AI".
- joxel 4y agoI mean no responsible developer would take the output of a ML model and then act on that without any tests on whether the output makes sense or not. That's the way to handle unexplainable AI. There's also opportunity to look at uncertainty estimation, i.e. look at the epistemic uncertainty of the model and use it as a proxy for potential error. That seems to be a main thrust for AI right now. I'm currently writing a paper looking at this for dynamical systems.
- mdp2021 4y ago> To me "Stakeholders want profit but researchers want progress". > how [vs] how reliable The problem of "how" is exactly relevant to "how reliable". Understanding the workings allows an insight to weaknesses.