5 ms·
Honestly, as someone that researches ML, this is my major concern. It isn't AGI that has the potential to kill us, it is current ML systems that can't handle OO
by godelski 4y ago
Honestly, as someone that researches ML, this is my major concern. It isn't AGI that has the potential to kill us, it is current ML systems that can't handle OOD data and engineers put it in place because "that's the user's fault." Same reason we have Teslas crashing. AI safety might talk about AGI a lot, but their main area of research is modern systems and concerns over that.
OOD data is really hard to deal with FWIW. But personally I don't feel confident that adding more matrix multiplies won't generalize in a way such that OOD isn't of major concern.
- salawat 4y agoBy OOD, do you mean Out Of Domain?
- jeabays 4y agoOut of distribution.
- etrautmann 4y agoNot the parent but presumable “out of distribution” meaning being faced with inputs outside of the statistics of the training data set.
- raegis 4y agoThese acronyms are destroying my will to live.
- throwaway894345 4y agoGenuine question: is Tesla’s autopilot crashing more often or more severely than human drivers?
- nerdawson 4y agoThere was a post recently with a video of a Tesla attempting to drive into the path of an oncoming train: https://youtube.com/watch?v=yxX4tDkSc_g https://youtube.com/watch?v=yxX4tDkSc_g It’s not the big mistake at the end that stands out to me but the sheer volume of mistakes it makes along the way. Edging forward at an intersection when there’s a red light for example. I don’t doubt that self driving tech will improve and be a safer alternative to a human driver eventually. It doesn’t seem like we’re there yet though.
- throwaway894345 4y agoYeah, I fully expect Autopilot to have different failure modes than human drivers, but what I’m interested in is the different fatality rates (deaths per hundred million miles, adjusted for different types of roads i.e., highway vs city streets). If Autopilot can save hundreds of lives annually to human-error mistakes like falling asleep at the wheel, etc but at the cost of one life annually due to obscure failure modes like driving toward a train, I maintain that we should not only allow Autopilot, but probably even mandate it on new vehicles. Sacrificing hundreds or thousands of lives annually because we don’t like the specific failure modes seems absurd. Of course, if it doesn’t save lives, then we should block its deployment on those grounds (but the particular kind of failure mode shouldn’t affect the calculus).
- NamTaf 4y agoTo some degree, that doesn't matter. An underlying feature of a competent approach to safety in design is that the design must take maximal ownership of eliminating risk to all people in all scenarios that can be reasonably expected to result from the design. The moment Telsa set expectations by proclaiming it as autopilot, they took the corresponding responsiblity to make sure it did not generate any scenarios which were unsafe. The moment they implemented features that allowed the attention of drivers to drift more than standard driving, they also took responsibility to make sure that the drifting attention of drivers did not place the system in an unsafe state. This same issue applies to touch-screen interfaces in modern cars. Drivers could always stare down at their radio when there were tactile knobs and dials, but touch-screen interfaces now expect that because they've eliminated tactile feedback. Telling drivers 'just don't look down' misses the point, because it's the responisbility of the car manufacturer to not create a system where that added safety risk is not controlled appropriately.
- ALittleLight 4y agoAGI alignment is a vastly bigger problem. Of course poorly built and deployed ML systems will kill and injure people - but these are tragedies of the kind humanity can endure and overcome and has overcome. Poorly aligned AGI is nothing less than the entire species at stake.
- godelski 4y ago> AGI alignment is a vastly bigger problem But also far less likely to happen anytime soon. A bigger danger is when someone thinks a machine is sentient or "semi-conscious" (whatever that means) and naively uses it to do tasks it shouldn't.
- ALittleLight 4y agoI don't think you nor anyone else knows when AGI is likely to happen. I also don't think that incorrectly believing a machine is sentient when it is not is a "bigger danger." Again, an improperly aligned AGI could kill the entire human race. I'm not sure what harm incorrectly believing a machine is sentient might do, but I don't think it would be as bad as human extinction or enslavement which are both real possibilities with AGI.
- josephcsible 4y agoYou seem to be comparing only the worst-case impact and not the probability. To see why that's fallacious, consider that an asteroid could also kill the entire human race, but nobody would agree that asteroids are more dangerous than drunk driving.
- ALittleLight 4y agoI think there's a high probability of AGI within a century. Surveys show most experts share that opinion. It's hard to know the probability that the AI will be misaligned - but currently we have no idea how to align it. It's also hard to say how likely a misaligned AI would be to cause extinction. However, we have no reason to think that either of those things are unlikely.
- dmix 4y agoAre there any other major examples of modern ML ethical issues besides some Tesla cars killing their drivers? Are ML driven robots in factories killing people or Something? Because I haven’t heard of anything else. The only other modern AI ethics stuff I hear about is making image generators more politically correct and maybe some criminal sentencing algorithms that are being misused (which isn’t really an AI ethics problem but a judicial procedural one).
- kthejoker2 4y agoI think everyone here is justified in saying "do your own research." Plenty of stories if you just Google a bit.
- dmix 4y agoI was mostly being facetious, I have done my research. I don’t really expect many legitimately good answers (although I’ll keep an open mind). I’m still waiting for those GPT-3 and deep fake horror stories we were warned about to come to reality. Maybe ML needs more script kiddy frameworks like KaliOS for these tools before the justifications to slow down R&D will finally be justified?
- godelski 4y ago> I’m still waiting for those GPT-3 and deep fake horror stories we were warned about to come to reality. I'm a bit confused. There's plenty of propaganda written by ML. Here's some deep fakes with respect to Ukraine[0][1]. Manufacturing robots kill people all the time[2][3]. They are weaponizing ML. Like specifically GPT-3? Probably not but people do use these to write tweets and short form things. [0] https://news.northeastern.edu/2022/04/01/deepfakes-fake-news-threat-democracy/ https://news.northeastern.edu/2022/04/01/deepfakes-fake-news... [1] https://www.npr.org/2022/03/16/1087062648/deepfake-video-zelenskyy-experts-war-manipulation-ukraine-russia https://www.npr.org/2022/03/16/1087062648/deepfake-video-zel... [2] https://www.theguardian.com/world/2015/jul/02/robot-kills-worker-at-volkswagen-plant-in-germany https://www.theguardian.com/world/2015/jul/02/robot-kills-wo... [3] https://www.techrepublic.com/article/robot-kills-worker-on-assembly-line-raising-concerns-about-human-robot-collaboration/ https://www.techrepublic.com/article/robot-kills-worker-on-a...
- imtringued 4y agoThis is a classic case of undefined behaviour or memory unsafety. Your mistakes can have an infinitely bad outcome but people blame the programmer even though there are memory safe languages. Yes they sacrifice efficiency but who the fuck wants to consider the billion potential ways of operating a robot in a physically unsafe way? This means we are going to have the equivalent of GC in robots that interact with other humans.