4 ms·

Every time I read an article/paper/etc about "AI Safety" it always rubs me the wrong way. I think the reason why is these articles always completely discount us

Every time I read an article/paper/etc about "AI Safety" it always rubs me the wrong way. I think the reason why is these articles always completely discount user agency, that is they assume you will ask the AI for help with a thing and will be completely at the mercy of what it outputs. IMHO I think that we should spend less time obsessing about "AI Safety" and more time educating users about the limits, pitfalls and drawbacks of using LLM's. The way I look at it is since AI models are trained on internet data, the same rule of "don't believe everything you read/see on the internet" should apply. Just because a layer of abstraction has been applied to that data does not mean that the rule no longer applies.

Chabsff 3y ago
From an industrial point of view, the entire point of AI (as opposed to "traditional" algorithms) is to transfer more agency from humans to machine than is feasible with current techniques. Trust and safety are currently the two main things blocking that, which is why all of these discussions are very often worded in a way to maximally appeal to the purse holders of these industries.
lainga 3y ago
Half of the commenters I read on HN make me discount user agency.
awildfivreld 3y ago
In a perfect world, I agree that the average user/developer would be conscious of this, and take everything from an LLM with a grain of salt. However, I've since come to realize that too many people either just won't care about the implications, or are oblivious of them. I've come to realize that, in practice, relying on the user behaving adequately is going to create too much damage to rely on.
drdrek 3y ago
Recently a small startup CEO I know used information he got from an AI model (Bard) to get a list of relevant conferences. They created marketing campaigns based on that information, he never double checked the output. The conference dates were totally wrong and money was wasted. Do not underestimate how fast people turn lazy.
AndrewKemendo 3y ago
There are ~200000 GPS navigation system induced accidents per year in the US [1] Here’s a short list of just recent ones: https://www.autoevolution.com/news/driver-claims-gps-navigation-sent-him-in-a-lake-so-he-obeyed-158385.html https://www.autoevolution.com/news/driver-claims-gps-navigat... https://www.autoevolution.com/news/driver-frozen-to-death-in-50c-weather-after-taking-shorter-google-maps-route-152969.html https://www.autoevolution.com/news/driver-frozen-to-death-in... https://www.autoevolution.com/news/couple-spends-24-hours-stuck-in-snow-after-blindly-following-google-maps-158377.html https://www.autoevolution.com/news/couple-spends-24-hours-st... [1] https://www.psychreg.org/gps-drives-crash/ https://www.psychreg.org/gps-drives-crash/
- cbozeman 3y ago
  All of this revolves around people doing stupid shit and not critically thinking. There's a price to be paid for being oblivious to your surroundings and not thinking. I have a niece who is 20 years old, scored very high on a formal IQ test, is autistic, and who has been in three car accidents - all her fault - because she doesn't pay attention. Now her insurance is $500 per month. And her Mom has cut her off and she's paying for it now. Life will make you learn to pay attention... the severity of the lesson is up to you.
  - AndrewKemendo 3y ago
    Bottom line: Failing to account for human fallibility and misuse in product design is dangerous and deadly I certainly agree that individual responsibility is paramount. The reality however is that “modern, connected, capitalist” life in 2024 requires you to have faith in thousands of systems that you don’t even know exist, are not designed the way I describe and are actually adversarial to the customer. As a designer, and engineer, I view it as my responsibility to deliver products that do not make customers worse (in any time horizon) as a result of using my product and to limit or prevent externalities that impact the systems that sustain the product and the customer holistically over the longest possible time horizon. However very few systems are built to such a customer-centric spec It shouldn’t require genius level intelligence to have a reasonable understanding of how the systems you rely on work and the impacts of them breaking. However millennia of specialization has ensured that the complexity and externalities of any one sub-system is undefined. So the concatenation of systems is a NP hard problem just to conceptualize. The fact that we don’t have defined system boundaries, a measurable goal state or current state means we’re a directionless set of agents susceptible to reward highjacking
oceanplexian 3y ago
Notice how they specifically use the word "open". I guess "closed" models aren't a problem, huh. You, peasant, can rent it by the token and maybe Big Tech will throw you a bone if you send it some queries approved by the content policy. But whoops, the cat got out of the bag. We can run decent models on a run of the mill gaming PC. China and many others are creating competitive models that are fantastic and open sourcing them. GPT4 is still the top dog, but OSS isn't far behind. A lot of very big, influential companies are facing some disruption. It's VERY obvious why safety is being pushed in tech media. Eventually as they get more desperate you'll learn exactly why they spend so much money on lobbyists when they try to make it illegal to run open source AI models.
benreesman 3y ago
AI Safety stuff rubs you the wrong way because it’s a disingenuous, hyperbolic, bad-faith power grab / stall tactic most of the time. It’s not always, but there’s no money in the honest stuff. The AI stuff is powerful, it stands to change which companies and people are rich and powerful, and like always, they want it dead or limited or constrained until they can control and monopolize and capture it. This is an old story even with names like Microsoft in the story. Asimov had proposed the 3 laws like, 70 or 80 years ago, describing machines far more powerful than any language model, all of humanity has had at least that long to debate and consider and discuss that, lots of people have, and “put some creepy, insular, privatized clique in Atherton in charge with zero oversight until we’re safe” was zero times on the menu. Asimov’s 3 laws still seem about right, and if they need updating? Not a private company that fires board members when the board tries to police the CEO. That needs to be the public’s consensus in one of the many ways the public weighs in on stuff.
- pixl97 3y ago
  >Asimov’s 3 laws still seem about right, [meme] My brother in Christ, what are you talking about! [end meme] The purpose of the books was to tell you the 3 laws did not work.
  - benreesman 3y ago
    It’s a bit of a cliffhanger whether the zeroeth law as interpreted by R. Daneel Olivaw and culminating in the choice of Gaia’s governance instead of either the First or Second Foundation is going to go well, but he died within months of writing that far. I’m not any kind of official or recognized Asimov scholar, just a fan, but I’ve read his prodigious catalog repeatedly and love a good “some is wrong on the Internet babe”. You seem to feel differently about the end state?
jasonjayr 3y ago
gestures exasperated at the last 25 years of media literacy, social media and tech literacy You're right, but it looks like the only way the general public want to do this is to restrict, lock down, rent seek, and keep putting up guard rails on technology to remove user agency from general purpose computing.
pixl97 3y ago
Computer says no. https://www.youtube.com/watch?v=0n_Ty_72Qds https://www.youtube.com/watch?v=0n_Ty_72Qds Computers don't argue. https://en.wikipedia.org/wiki/Computers_Don%27t_Argue https://en.wikipedia.org/wiki/Computers_Don%27t_Argue ----- The problem here is you want to make a Moloch problem an individual problem... This doesn't always work, people will defer to the system and not take self responsibility, especially when the incentives align to defer.
deleted 3y ago
[deleted]
ozr 3y ago
I agree, but the push for 'AI Safety' as a whole is driven by the x-risk crowd. There are a few people genuinely concerned, however misguidedly, about the outputs of LLMs. The big money coming into the field from EAs and the sort of background fear is about fast takeoff, shoggoths, paperclip maximizers, etc. The thought is that, if you can't reliably align an LLM to output what some authoritative source wants, then you can't reliably align the inevitable machine god that will destroy us. In that context, user education doesn't do any good.

Chabsff 3y ago

From an industrial point of view, the entire point of AI (as opposed to "traditional" algorithms) is to transfer more agency from humans to machine than is feasible with current techniques. Trust and safety are currently the two main things blocking that, which is why all of these discussions are very often worded in a way to maximally appeal to the purse holders of these industries.

lainga 3y ago

Half of the commenters I read on HN make me discount user agency.

awildfivreld 3y ago

In a perfect world, I agree that the average user/developer would be conscious of this, and take everything from an LLM with a grain of salt. However, I've since come to realize that too many people either just won't care about the implications, or are oblivious of them. I've come to realize that, in practice, relying on the user behaving adequately is going to create too much damage to rely on.

drdrek 3y ago

Recently a small startup CEO I know used information he got from an AI model (Bard) to get a list of relevant conferences. They created marketing campaigns based on that information, he never double checked the output. The conference dates were totally wrong and money was wasted. Do not underestimate how fast people turn lazy.

AndrewKemendo 3y ago

There are ~200000 GPS navigation system induced accidents per year in the US [1] Here’s a short list of just recent ones: https://www.autoevolution.com/news/driver-claims-gps-navigation-sent-him-in-a-lake-so-he-obeyed-158385.html https://www.autoevolution.com/news/driver-claims-gps-navigat... https://www.autoevolution.com/news/driver-frozen-to-death-in-50c-weather-after-taking-shorter-google-maps-route-152969.html https://www.autoevolution.com/news/driver-frozen-to-death-in... https://www.autoevolution.com/news/couple-spends-24-hours-stuck-in-snow-after-blindly-following-google-maps-158377.html https://www.autoevolution.com/news/couple-spends-24-hours-st... [1] https://www.psychreg.org/gps-drives-crash/ https://www.psychreg.org/gps-drives-crash/

cbozeman 3y ago
All of this revolves around people doing stupid shit and not critically thinking. There's a price to be paid for being oblivious to your surroundings and not thinking. I have a niece who is 20 years old, scored very high on a formal IQ test, is autistic, and who has been in three car accidents - all her fault - because she doesn't pay attention. Now her insurance is $500 per month. And her Mom has cut her off and she's paying for it now. Life will make you learn to pay attention... the severity of the lesson is up to you.
- AndrewKemendo 3y ago
  Bottom line: Failing to account for human fallibility and misuse in product design is dangerous and deadly I certainly agree that individual responsibility is paramount. The reality however is that “modern, connected, capitalist” life in 2024 requires you to have faith in thousands of systems that you don’t even know exist, are not designed the way I describe and are actually adversarial to the customer. As a designer, and engineer, I view it as my responsibility to deliver products that do not make customers worse (in any time horizon) as a result of using my product and to limit or prevent externalities that impact the systems that sustain the product and the customer holistically over the longest possible time horizon. However very few systems are built to such a customer-centric spec It shouldn’t require genius level intelligence to have a reasonable understanding of how the systems you rely on work and the impacts of them breaking. However millennia of specialization has ensured that the complexity and externalities of any one sub-system is undefined. So the concatenation of systems is a NP hard problem just to conceptualize. The fact that we don’t have defined system boundaries, a measurable goal state or current state means we’re a directionless set of agents susceptible to reward highjacking

oceanplexian 3y ago

Notice how they specifically use the word "open". I guess "closed" models aren't a problem, huh. You, peasant, can rent it by the token and maybe Big Tech will throw you a bone if you send it some queries approved by the content policy. But whoops, the cat got out of the bag. We can run decent models on a run of the mill gaming PC. China and many others are creating competitive models that are fantastic and open sourcing them. GPT4 is still the top dog, but OSS isn't far behind. A lot of very big, influential companies are facing some disruption. It's VERY obvious why safety is being pushed in tech media. Eventually as they get more desperate you'll learn exactly why they spend so much money on lobbyists when they try to make it illegal to run open source AI models.

benreesman 3y ago

AI Safety stuff rubs you the wrong way because it’s a disingenuous, hyperbolic, bad-faith power grab / stall tactic most of the time. It’s not always, but there’s no money in the honest stuff. The AI stuff is powerful, it stands to change which companies and people are rich and powerful, and like always, they want it dead or limited or constrained until they can control and monopolize and capture it. This is an old story even with names like Microsoft in the story. Asimov had proposed the 3 laws like, 70 or 80 years ago, describing machines far more powerful than any language model, all of humanity has had at least that long to debate and consider and discuss that, lots of people have, and “put some creepy, insular, privatized clique in Atherton in charge with zero oversight until we’re safe” was zero times on the menu. Asimov’s 3 laws still seem about right, and if they need updating? Not a private company that fires board members when the board tries to police the CEO. That needs to be the public’s consensus in one of the many ways the public weighs in on stuff.

pixl97 3y ago
>Asimov’s 3 laws still seem about right, [meme] My brother in Christ, what are you talking about! [end meme] The purpose of the books was to tell you the 3 laws did not work.
- benreesman 3y ago
  It’s a bit of a cliffhanger whether the zeroeth law as interpreted by R. Daneel Olivaw and culminating in the choice of Gaia’s governance instead of either the First or Second Foundation is going to go well, but he died within months of writing that far. I’m not any kind of official or recognized Asimov scholar, just a fan, but I’ve read his prodigious catalog repeatedly and love a good “some is wrong on the Internet babe”. You seem to feel differently about the end state?

jasonjayr 3y ago

gestures exasperated at the last 25 years of media literacy, social media and tech literacy You're right, but it looks like the only way the general public want to do this is to restrict, lock down, rent seek, and keep putting up guard rails on technology to remove user agency from general purpose computing.

pixl97 3y ago

Computer says no. https://www.youtube.com/watch?v=0n_Ty_72Qds https://www.youtube.com/watch?v=0n_Ty_72Qds Computers don't argue. https://en.wikipedia.org/wiki/Computers_Don%27t_Argue https://en.wikipedia.org/wiki/Computers_Don%27t_Argue ----- The problem here is you want to make a Moloch problem an individual problem... This doesn't always work, people will defer to the system and not take self responsibility, especially when the incentives align to defer.

deleted 3y ago

[deleted]

ozr 3y ago

I agree, but the push for 'AI Safety' as a whole is driven by the x-risk crowd. There are a few people genuinely concerned, however misguidedly, about the outputs of LLMs. The big money coming into the field from EAs and the sort of background fear is about fast takeoff, shoggoths, paperclip maximizers, etc. The thought is that, if you can't reliably align an LLM to output what some authoritative source wants, then you can't reliably align the inevitable machine god that will destroy us. In that context, user education doesn't do any good.