19 ms·
For Tesla, Facebook and Others, AI’s Flaws Are Getting Harder to Ignore
- gumby 5y ago“AI” is just a buzzword name for “software” these days
- machinelearning 5y agoQuestion for people here: Is it justifiable to accept these flaws in the short-term if it results in a car that has a lower-error rate than human drivers in the medium/long-term?
- gameswithgo 5y agoThere is no need for us to accept the flaws in the short term. People can keep developing this tech without us mass adopting it first.
- machinelearning 5y agoI considered this before posting, but I think that human disengagements are an important supervisory signal for the model. You can’t hire enough safety drivers to scale this process and releasing it is the fastest way to get there. Simulations are not going to get you all the way because you can’t simulate the real world perfectly. Do you have any thoughts on how to develop this without releasing it?
- DoingIsLearning 5y ago> You can’t hire enough safety drivers to scale this process and releasing it is the fastest way to get there. I am sorry but I sincerely hope you are never made responsible for the release of anything remotely safety-critical. Please re-read what you wrote. You are saying that because a business does not have enough money/resources to scale a process in real-world conditions that the solution is to release something and verifying in the real-world, on a public road, risking real human lifes? If you don't have enough money to test it then you don't have an actual product/business.
- whimsicalism 5y ago> I am sorry but I sincerely hope you are never made responsible for the release of anything remotely safety-critical. Wild to infer that from someone asking a hypothetical ethical question. Why are so many on this site so aggro?
- DoingIsLearning 5y agoHad to google aggro, no aggression intended at all, but I frankly found the severity of GP question too scary to be considered. > I considered this before posting ... To me this is not someone asking a hypothetical but someone who is aware that what they are about to write is controversial yet are still considering it to be the best way forward. I happened to strongly disagree with that.
- whimsicalism 5y agoIf there is clear evidence that these cars would have lower fatality rates than human drivers (I don't think this evidence currently exists), then holding off release to continue development is potentially a moral bad, for the same reason that we end clinical trials early when they demonstrate clear life-saving potential that would save lives in the control group. I don't think it is morally reprehensible to ask what other people's intuition around this problem is.
- sharkjacobs 5y ago> Is it worth killing n random people, for a uncalculated and unknowable chance of saving n+m lives in the future. The pool of people randomly selected to die have not volunteered or consented to be part of this project. There are ways to achieve the same lifesaving endgoal without the upfront sacrifice of lives. The part which really strikes me as morally reprehensible is where the companies are saving money on test drivers and controlled test environments and externalizing those costs onto every other driver sharing the road with their training data collectors
- kevingadd 5y agoWaymo appears to disagree with you, they ran tests with safety drivers and in controlled conditions for a very long time, and they eventually got to the point where there are customers riding in unmanned taxis. Tesla is just applying a "move fast and break things" approach to pedestrians' lives in order to satisfy shareholder interests and executive egos. They're not the only ones, as we saw from Uber's absolutely shameful performance that eventually lead to a death (safety driver involved but not solely responsible, of course)
- KallDrexx 5y agoI was under the impression that while Waymo did some tests unsupervised, most of the rides that non-employees can hail still have a supervisor. Is that not true?
- kevingadd 5y agoI've seen youtube videos of unsupervised waymo cab rides, and when it got stuck in one they sent techs out to rescue it. It's probably still uncommon.
- lvl100 5y agoThis comparison gets me the most. People bringing up human error rate vs machine error rate. They’re not comparable at all. And it will be evident when you have majority autonomous cars on the road. Human errors are more or less RANDOM. Machine errors are NOT.
- machinelearning 5y ago1. I’d challenge the premise that human errors are random. There are a ton of patterns that cause accidents including intoxication, low visibility conditions and tiredness. I haven’t done a statistical analysis but I’d hazard a guess that only a minority of human accidents are truly random. 2. Why does the randomness matter if the error rate is lower? Certainly if the errors are predictable, they can be discovered and fixed or avoided?
- grlass 5y agoalso, human failure modes are better understood, and can be better anticipated by e.g. you can still have a somewhat predictive mental model of how a swerving drunk drive might behave. We don't have a good frame of reference for how machines might behave with their failures, which means that accidents could be worse than they would be otherwise.
- machinelearning 5y agoThe severity of the accident is an interesting point. Though it intuitively feels like there are ways for software to mitigate the severity of an accident when it realizes it is about to crash than a human who might be asleep, intoxicated or otherwise have a slower reaction time.
- kevingadd 5y agoThe machine's ability to recognize that it's about to crash may actually be one of the issues here, since often the self-driving/driving-assist car crashes are cases where the AI just completely misinterpreted the environment and made bad choices. A human driver is somewhat likely to eventually realize what situation they've gotten themselves into (oh no, i can't stop in time) because of the multiple different feedback loops and information sources they're working with combined with their experience as a driver. For example, a drunk or very tired driver is operating with impaired decision making and response time, but they may eventually notice and respond - while an AI misclassifying a fire truck as a stop sign may very well continue misclassifying it until impact. One way to mitigate this would be via sensor fusion - even if your vision or radar sensing fail, you can rely on data from other sensors to do things like apply emergency braking. Unfortunately at least one vendor has decided to ditch radar, lidar, etc and just go with vision!
- ClumsyPilot 5y agoThe same argument would justifies human medical experiments, and we know they actually work, and result in faster access to improved drugs and vaccines. Unlike autopilot which could still be crappy after 10 years Yet we have made them illegal after some rather nasty precedents. Seems like hsitory repeats itself
- RandallBrown 5y agoI think the difference is that human medical experiments immediately cause suffering at a higher rate than whatever disease they're trying to fix. I believe that there have been studies showing that today's "self driving" cars are already statistically safer than regular cars.
- Forge36 5y agoIt's about risk trade off. If it's 2x better than humans is it worth it? 10x? 100x? 36,096 deaths in 2019 in U.S. ~1.3 million worldwide (I couldn't find injury statistics this morning) If the flaw is found before someone dies from it I'm not concerned. If 1 person dies instead of 10 I'm all for it. (I'd take 2x better than humans any day)
- mplewis 5y agoWe can’t ignore core usability and basic safety issues by saying “on average, this is better.” End users can’t be expected to know that they’ll probably be fine, but an edge case they don’t understand will kill them one evening when they drive past a stopped ambulance.
- Forge36 5y agoWhy not? If it saves 30,000 lives per year is that not a meaningful improvement? I suspect my understanding/acceptance of "unknown risk while driving" is flawed in some way, or at least very different from the general populace.
- tragomaskhalos 5y agoThe problem is, it's not a level playing field. 100 incidents of a person ploughing into a bus queue and killing a child, each is news for a day, everyone accepts the tragedy and moves on. A self-driving car does it once though, and the mob will be at the factory gates with torches and pitchforks.
- Forge36 5y agoThat's an interesting problem. I wonder what social discussions need to take to reach acceptance for "AI assisted driving". Mandatory AI breaking to prevent driving into a bus queue? I'm also curious how to find the people who do object vs theory-crafting all possible concerns people could have.
- SuoDuanDao 5y agoHm. I find this very analogous to the MRNA vaccine debate: how high an error rate of a new technology do we accept, and to what degree does that choice have to be made at a community rather than an individual level? I'd feel best if that decision was made at the smallest community level possible, so ideally county by county rather than federally. That lightens the burden of politicians making the wrong choice or being a citizen who disagrees with the right choice.
- whimsicalism 5y agoYou get far more localized really shitty situations if you devolve to that level. Sometimes central decision making is good. Having to switch the mode of operation of your car depending on what side of various county lines you are on seems like an obvious regulatory failure.
- SuoDuanDao 5y agoThe error rate would certainly go up at first due to the additional complexity, but the cost of each error goes down. Besides, I have to switch the speed at which I'm driving far more granularly than at county lines, and no one's worried about a car's ability to handle that. I don't disagree that sometimes central decision making is good, but in a complicated situation where any decision will have some negative consequences depending on the specific context seems like a textbook case of not being one of them.
- tdrdt 5y agoIt depends on what kind of error it produces. If the error rate is only 0.5% but the death rate of those errors is 100% I am not sure it will be justifiable.
- aspaceman 5y agoEven if the death rate is low but the _perceived_ impact is high. Ex: Imagine a week where self-driving cars have a bug that only mis-identifies grandmas. Only a few grandmas die but the perceptual impact is massive.
- klyrs 5y agoNo, absolutely not. I'd say that Tesla flirts with the opposite outcome, that we saw with nuclear power. Early catastrophic failures resulting from premature deployment can produce a very reasonable revulsion away from the technology, such that the technology will have a PR problem well past the "break even" point. That folks continue to be bullish on self-driving tech despite cars repeatedly hitting stopped emergency vehicles at speed tells me that tech enthusiasts haven't learned from history.
- postalrat 5y agoThe problem nuclear power has had is nuclear reactors are a great tool to build materials for a nuclear bomb. If everyone accepted them then access to nuclear weapons would have been a lot easier.
- mjohn 5y agohttps://archive.is/FFrU5 https://archive.is/FFrU5
- danschumann 5y agoHarder to debug and harder to fix
- amelius 5y agoYes, would you trust a car with an error rate of 0.001%?
- CJefferson 5y agoThe question is, what do you mean by "trust"? Would I be happy for it to be driving around on the road? Probably. Would I be happy for it to drive me, and it's 'my fault' if I don't notice it's gone wrong and kill someone? No. So far Tesla (for example) seems nowhere near the point where they would accept responsibility for crashes -- they still always blame the driver for not paying attention.
- bsenftner 5y agoThat's the key tell right there: Musk always blames the customer and never takes responsibility.
- rpadovani 5y agoWell, what's the baseline? I'm all in for a way to reduce any accident.
- LightG 5y agoBut there's a choice involved. Reduce baseline accidents, but increase being exposed to completely random events that will kill you. Nope. I won't be part of that statistic.
- kemitche 5y agoWe make those choices all the time. Taking a plane instead of driving is similar. You lose direct control, but it's orders of magnitude safer. I'll take a 0.001% chance of death over a 0.1% chance every time.
- LightG 5y agoI'll take 0.1% chance of death which I (arguably) can control, over a 0.001% chance of death which I absolutely cannot control at all. It's so clear to me. Yes, taking a plane is similar, but I only take that risk a couple of times a year. The 0.1% (to me) is a probability number as I have some control over each event. The probability is so low that it likely never will happen. The 0.001% number is an eventual outcome number. "Run the experiment X amount of times, and death will happen 0.001% of the time". And it's more relevant than flying as we drive so much more. Nope! Thanks!
- amatix 5y agoAnd at the same time, the UK government is looking at reversing the GDPR review right for automated decisions: Article 22 guarantees that people can seek a human review of an algorithmic decision, such as an online decision to award a loan, or a recruitment aptitude test that uses algorithms to automatically filter candidates. In May, a government task force set up to look for deregulatory dividends from Brexit, led by the leading Brexiter Iain Duncan Smith, argued that Article 22 should be removed because it made it “burdensome, costly and impractical” for organisations to use AI to automate routine processes. The idea is part of broad-based plans for a big overhaul of the UK data regime after Brexit which ministers say will boost innovation, and deliver what Oliver Dowden, the culture secretary, has called a “data dividend” for the UK economy. https://www.ft.com/content/519832b6-e22d-40bf-9971-1af3d3745821 https://www.ft.com/content/519832b6-e22d-40bf-9971-1af3d3745... (Edit: formatting/link)
- te_chris 5y agoThese people are such chumps. Automated hellscape for thee, but not for me.
- andrepd 5y agoOh they are not chumps. They know exactly what they're doing.
- flipbrad 5y agoI advise companies on GDPR compliance, and art 22 is usually the least of their concerns. Why not offer a right to human review, if your algorithm is producing "legal or similarly significant effects"? If you're not 100% convinced in the accuracy and fairness of your automated system (two entirely separate GDPR rules), you can avoid issues down the line by offering individuals the ability to flag a dodgy decision and have a human look into it.
- tomp 5y agoIs this another one of those GDPR articles that has no teeth? I cannot imagine how Google can keep running it’s spam filters, Facebook it’s automated bans, and Twitter its algorithmic feed, while abiding by this.
- nikkinana 5y agoIt's pretty easy to ignore Bloomberg, they're a bunch of dopes.
- amelius 5y agoWhat will happen if a deep-learning bug is found in Tesla autopilot where it misclassifies a fire truck for a bridge? Will they ground all the Tesla cars remotely? Or disable autopilot remotely? Until they have gathered new training data and updated the software?
- drcode 5y agoThe same thing that happens to a human that is at risk of misclassifying a fire truck for a bridge: Once the odds of this happening are low enough (say 0.00001%) they can pass a driver's test and we give them a drivers license.
- tguedes 5y agoThe problem is that's 1 person. For Tesla's autopilot, that's the hundreds of thousands of cars running that broken model. The scale of the problem is much larger.
- Iolaum 5y agoTo add context. In Aviation, if they do find a bug in the software running an airplane's engine they do ground planes until they know more about impact and mitigation. Not saying cars should do the same, just that it's not absurd to consider it.
- hef19898 5y agoWell, to be honest something like Tesla's Autopilot would never have been certified in Aerospace anyway. And yes, the question of what authorities would do in such a case is a valid one. It seems like authorities are struggling with functions like that, would it be simply hardware, e.g. faulty airbag sensors, recalls would have been already be issued.
- ClumsyPilot 5y ago" It seems like authorities are struggling" I would rather describe it as sleeping at the wheel, passed out drunk and having pissed their pants after vomiting a bottle of hard liquir. Okay, maybe i got carried away with the metaphor, but you get the idea
- DrNuke 5y agoPoint is, we/they all knew the unavoidable AI’s flaws way before the s*it hit the fan, the underlying sci-fi assumptions undermining the science… and the problem is, some states’ wild de-regulation allowing deployment in public, commonal spaces (real and virtual) being managed like a gigantic theme park and joe mcjoes becoming the guinea pigs for such, well-advised experiments.
- rootusrootus 5y agoI've lost count of the number of people I've talked to who think that neural nets means we've created brains that will just magically learn how to do new tasks. So we just need more training and then automated <whatever> is just around the corner. Tesla, et al do not have the luxury of ignorance to explain that away however, they know what the technology is and is not currently capable of, but they don't want to admit it.
- gibsonf1 5y agoIts all artificial but no Intelligence. Statistical pattern matching, no matter how sophisticated, does not understand "why" from conceptually thinking about a space/time model of the world as we humans do.
- ChefboyOG 5y agoEven if this is the case, why does it matter? If the explicit goal is to create a human intellect, then sure, there's a really interesting conversation there—one that is happening constantly in the DL/AI research community, in which virtually no one believes that we're close to AGI or that current deep learning is going to achieve it. But that's explicitly not the goal that 99.9% of neural networks are designed with. Their traditional use case is where they excel: programmatically approximating functions that are exceedingly hard to approximate manually. This includes but is not limited to image recognition, speech synthesis, recommendation (including search), fraud detection, ETA prediction, even medicinal chemistry.
- buitreVirtual 5y agoEntirely agree. The problem is that most people don't understand that and easily fall prey to thinking that AI is a magical black box that can solve any problem you throw at it. In no small part because of all the hype by salespeople and the media. The reality is that NNs are great but only for some types of problems AND where there is a high tolerance to false positives and negatives. Clearly, this does not include problems where safety is critical (unless you can really demonstrate that AI does better at safeguarding than humans AND you can convince the public to not be scared).
- tdrdt 5y agoIt seems the flaws are too binary. 90% of the results are great, the other 10% absolute trash. You can see this everywhere. For example those app that generate a non existing person. A lot of times the results are great except for that one spot which makes the overall result useless. Another example is the OptiX denoiser (NVidia). You can get very nice renders in a few seconds which speeds up the workflow. But every time it has areas with a lot of flaws. This doesn't matter when you are still working on something but for production it is useless. ML has it's use in a lot of areas where the outcome doesn't have to be perfect. But I am still not convinced it is 'production ready'.
- michaelbuckbee 5y agoThere's the notion that over time the flaws of the previous generation of technology are held up as traditional artifacts. Things like: - the "warm" sound of vinyl records. - nostalgia for early myspace, tumblr, geocities, web design - faux edison lightbulbs - low vs high frame rate movies I wonder if the flaws of all the current ML techniques will eventually be thought of similarly.
- gwern 5y ago> Businesses are also shifting their focus away from “AI-as-a-service” vendors who promise to carry out tasks straight out of the box, like magic. Instead, they are spending more money on data-preparation software, according to Brendan Burke, a senior analyst at PitchBook. He says that pure-play AI companies like Palantir Technologies Inc. and C3.ai Inc. “have achieved less-than-outstanding outcomes,” while data science companies like Databricks Inc. “are achieving higher valuations and superior outcomes.” Palantir is now a "pure-play AI company"? (And, for that matter, a market cap of $50b is 'less than outstanding'?)
- gipp 5y ago> And, for that matter, a market cap of $50b is 'less than outstanding'? Less than outstanding outcomes Market cap is their outcome, not their clients' outcomes. The two are decidedly different things, especially in our weird distorted market.
- gwern 5y agoBurke specifies "valuations" as well as outcomes, so the observation stands. (Not that Palantir is any kind of monopoly... If you will recall, the usual criticism is that it's just a consulting shop with zero moat or monopoly.)
- 6gvONxR4sf7o 5y agoI am starting to wonder if we’re pretending to live a little further into the future than we realistically can right now. Modern Silicon Valley is built around businesses with software margins. If you can scale via software, it’s huge. If we actually saddled these companies with their externalities or with doing what they say they do, would they still have been able to have software margins, or would they have had to wait until the tech/science was better? It gives me a wary feeling when people talk about tech regulation and warn that it would change the internet as we know it. Like, if putting the externalities on the company means the company can’t exist as it does today, is that really so bad?
- txbuck 5y ago"For Tesla, Facebook and Others, Misusing AI and Pretending It's a Magic Bullet Is Getting Harder To Ignore"