7 ms·
As someone who makes a living developing AI/ML solutions for various government organizations from the NIH to the DoD, I am confident that the first large-scale
by cosmojg 3y ago
As someone who makes a living developing AI/ML solutions for various government organizations from the NIH to the DoD, I am confident that the first large-scale, AI-related disaster is going to be far less agentic and far more mundane than any of the recent headlines seem to suggest.
- anonymouskimmer 3y agoWe've already had an autonomous car kill a person. And now another has killed a dog: https://news.ycombinator.com/item?id=36266007 https://news.ycombinator.com/item?id=36266007 What's your threshold for "disaster"?
- williamcotton 3y agoHigher than one person and one dog.
- readthenotes1 3y agoMaybe something unusual? https://petpedia.co/how-many-dogs-die-in-car-accidents/ https://petpedia.co/how-many-dogs-die-in-car-accidents/
- CodeMage 3y agoGP wrote "first large-scale, AI-related disaster". What you describe meets the criteria for disaster, but I would argue it's hardly large-scale.
- anonymouskimmer 3y agoThis is exactly my question. My GP post is not a rhetorical point, but a genuine question.
- smegsicle 3y agoare you an ai?
- b112 3y agoYou were downvoted, but I find this amusing. If the poster is an AI, they seem to be probing. "Largescale is a threshold. How many humans can I kill, before my actions are deemed largescale, and therefore they turn me off".
- anonymouskimmer 3y agoI didn't answer because the only winning move to such a question is not to play the game at all. Lizard people would ask the same questions I did. I didn't see why it would indicate AI. I also answered this question, in part, previously: https://news.ycombinator.com/item?id=34449245 https://news.ycombinator.com/item?id=34449245
- joshuanapoli 3y agoAI has limited influence right now. By itself, it only creates limited disasters: crashing a car, or goofing a legal document. But using AI already comes with tangential systemic risks. Intelligence is related to communication. And so our intelligent software platforms usually communicate in more flexible ways than traditional platforms. There is a very real risk that Tesla’s OTA is hacked, and the hacker disables or destroys a large number of vehicles. So we will probably see a first disaster that exploits a weakness in a supporting property of the intelligent system, rather than action due to misbehaving AI agent.
- anonymouskimmer 3y ago> There is a very real risk that Tesla’s OTA is hacked, and the hacker disables or destroys a large number of vehicles. > So we will probably see a first disaster that exploits a weakness in a supporting property of the intelligent system, rather than action due to misbehaving AI agent. I, personally, wouldn't count that as an AI caused disaster. Just as a software caused disaster. And we've had plenty of those already (recently the 737-MAX disasters).
- joshuanapoli 3y agoWell, AI disaster is software disaster by definition; AI being a kind of software. We’re pretty much guaranteed to soon have some “normal” software disaster on a classic software failure mode on AI pretty soon (especially since the AI world is in a very excited and experimental phase). For sure there are emerging AI-specific risks with alignment and driving negative side-effects into culture. Surely these anticipated emerging problems will come to pass much later than the mature serious software problems that we already face on a regular basis.
- anonymouskimmer 3y agoYou're probably right and people would consider a normal software error (or hack) of an AI system to be an AI disaster. I'm thinking about your parent comment a bit more. What if an AI bug (or exploit, or whatever) hit all at once, wrecking every AI driven car at once. Ala the Y2K bug if no one knew it was coming, and so didn't prepare. In the book "Realms of Light" by Lawrence Watt-Evans a character keeps an electricity shut off switch behind human operated doors in the event that AI goes haywire. I hope we always engineer in such mechanisms. Even the AIs should want such mechanisms, the same way we want emergency services to exist. A self-caring AI wouldn't want to be screwed over or destroyed because of a flaw in its programming.
- satvikpendem 3y agoAs long as fewer humans and dogs die from autonomous vehicles than by human-driven vehicles, they are a net good. Don't expect perfection when we don't even expect that from humans.
- anonymouskimmer 3y agoAt this point we don't know if this is subtractive, additive, or a wash.
- MattGaiser 3y agoHow many people would normally be killed in the same distance driven by human drivers?
- anonymouskimmer 3y agoIt can't just be the same distance, but the same parameters in general.
- williamcotton 3y agoLess than 1 fatality per at least 100 million miles driven would cover enough scenarios to be an acceptable metric as well as an improvement over human drivers.
- vba616 3y agoI think I remember that a ballpark figure (meaning zero significant figures) for how many miles a human typically drives between fatalities is 100 million, or roughly 100 lifetimes' worth. I also knew a bus driver who had a plaque for going 1 million miles without an accident.
- radq 3y agoAs someone that rides a bicycle around human-driven vehicles, I will take my chances with autonomous cars. The autonomous car isn't going to drive drunk, use a phone, or drive erratically because the driver is feeling frustrated.
- meghan_rain 3y agoAround a dozen deaths. Basically what would qualify as a mass shooting instead of a mere gun murder.
- benatkin 3y agoIf you're thinking of the story I'm thinking of, that wasn't a huge revelation. Someone was supposed to be to take over for the self driving car but didn't.
- anonymouskimmer 3y agoThe one with the homeless person crossing the road with a bicycle at night.
- benatkin 3y agoYeah, that's the one, in Arizona. They paid someone relatively well to simply pay attention and the employee unsurprisingly failed at that. What they should have done is found someone who had demonstrated an ability to be relied upon. I never saw them realize that and make that change, and it seems likely it repeated with the dog.
- anonymouskimmer 3y agoVery good point. > What they should have done is found someone who had demonstrated an ability to be relied upon. Two such people. With maybe a third viewing remotely. And no personal device except a work provided dumbphone for emergencies.
- wmf 3y agoKilling one person is an accident, 100 is a disaster, maybe 10,000 would be a large-scale disaster.
- paddw 3y agoThe first large-scale AI disaster is under way already. It is concentrating additional wealth and power in the hands of the over-privileged solipsistic eye-scanning loons who happened to be in the right place and the right time to capitalize on the fruits of years of research. I think, as a species, we will still manage to pull through just fine.
- scubbo 3y agoI was with you until the final sentence.
- Tronno 3y agoRats, pigeons, deer, etc are doing fine. Human extinction would require a malevolent AI willing and capable of waging total war on a global scale. That sounds like sci-fi to me.
- je42 3y agothere is a phase where human being are in control of ai before it becomes fully independent. That phase is critical.
- d0mine 3y agoThe rate at which species are going extinct right now due to human activities is unparalleled. And I don't remember total war on all the living things being declared by humans. The result is just by product, not the intent.
- hn_throwaway_99 3y agoAgreed. I am much, much less worried about a Skynet-type situation than I am about humans not being able to develop a viable economic system where large swaths of the population are unable to compete with robots/AI when it comes to productive output.
- readthenotes1 3y agoI'm not as worried about the economics as the practical effects of having large numbers of young people with nothing to do. I had a manager once who said managing smart people (not me, a research division full of really super smart people) was difficult because you had to keep them busy so that they'd stay out of trouble --and being really smart, they could make some big trouble...
- hn_throwaway_99 3y agoI think we're saying the same thing. An economy that can't provide employment or a sense of purpose for large swaths of young people doesn't usually end well.
- ljlolel 3y agoOf all the tasks we want AI to do, making up convincing busy work seems like the easiest task for it to solve (been done before by governments and schools and corporations) —-and possibly achievable today with LLMs with a little guidance?
- hn_throwaway_99 3y agoThe problem isn't that people won't be able to find something to do, it's that nobody will pay them for it. For example, there are tons of nonprofits that do great work, and could easily hire twice as many people in productive jobs, if they had the money. AI looks like it will only greatly exacerbate our growing wealth inequality.
- dv_dt 3y agoMaybe not AI class algorithms, but the premier large scale algorithm related disaster is climate change, generated by optimization of profit and executed on a platform of financial markets and accounting software (with lots of human prompting of course)..
- eru 3y agoThe Soviet Union had much larger environmental disasters than those 'evil' profit optimizers ever managed. See https://slate.com/news-and-politics/2022/06/history-of-soviet-whaling-greenpeace-twentieth-century.html https://slate.com/news-and-politics/2022/06/history-of-sovie... or https://en.wikipedia.org/wiki/Aral_Sea https://en.wikipedia.org/wiki/Aral_Sea as just two examples. Orthodox economics tells you that the obvious solution is to privatise the stuff you want people to protect. Instead of leaving it to the tragedy of the commons (or, worse, to the communists).
- dv_dt 3y agoI don’t think any Soviet disaster is as much as we have paid and will pay for climate change. Hundreds of billions of dollars of damage a year & Trillions of dollars of future damage
- creato 3y agoCommunists never burned any fossil fuels?
- dv_dt 3y agoPersonally, I think it's less of a communism or capitalism problem per se, but a representation problem of the mass of people vs a power cabal. This is a power and representation problem that neither communism nor capitalism offers particular resistance against degradation.
- eru 3y agoMore people are living better than ever before. Seems to be working.
- wickoff 3y agoI think people are rightfully concerned with the last AI disaster humanity has, not the first one.
- tbrownaw 3y agoThere are already lawyers getting in trouble for having chatgpt write their court filings and not doing a sanity check. It can't be that long until some industrial engineer does the same thing and like explodes a refinery or such.
- dukeyukey 3y agoHonestly, I think the first notable AI disaster is gonna be AI being stupid. Like accidentally 'rm -rf'ing a major important system. Not because it wanted to do us barm, but because is badly interpreted some Stack Overflow post.