6 ms·

First they rushed a model to market without safety checks, and I said nothing. It wasn't my field. Then they ignored the researchers warning about what it coul

First they rushed a model to market without safety checks, and I said nothing. It wasn't my field. Then they ignored the researchers warning about what it could do, and I said nothing. It sounded like science fiction. Then they gave it control of things that matter, power grids, hospitals, weapons, and I said nothing. It seemed to be working fine. Then something went wrong, and no one knew how to stop it, no one had planned for it, and no one was left who had listened to the warnings.

hsbauauvhabzb 8mo ago
Plenty of people have said plenty. The problem isn’t the warnings, it’s that people are too stupid and greedy to think about the long term impacts.
- ifh-hn 8mo ago
  Maybe it's how blunt this comment is that gets it downvoted, but I don't disagree.
  - hsbauauvhabzb 8mo ago
    I’ve noticed anti-AI stance gets downvoted on HN (and any anti-authoritarian comments, for that matter)
  - brookst 8mo ago
    No, it’s because it shows either a simplistic or needlessly confrontational view of the world. Unless you’re independently wealthy (as some in HN are), you have to balance your morals, your views of how things should work, feeding your family, and recognizing that you may not actually know everything. It’s easy to sit back and advise others that they should die on every single hill. But it’s not especially insightful, and serves mostly to signal piety rather than a well thought out view.
    - hsbauauvhabzb 8mo ago
      Spoken like a true LLM.
    - kakacik 8mo ago
      I am pretty sure a lot of horrible things were performed by rather regular folks with similar logic, don't need to invoke some WWII nazi extermination guard reference at all. Slippery slope, death by 1000 cuts and other synonyms describing exactly this.
    - ifh-hn 8mo ago
      Piety? To who? Simplistic and/or confrontational doesn't mean wrong, even if you don't like the way it's presented. Just because a comment is short, sharp, and to the point doesn't mean the author hasn't thought out why that's their view. No one knows everything, that's certainly why I'm on hacker news. I'm here to learn and expand my knowledge. Unfortunately a lot of people on here would rather driveby-downvote than have a discussion to find out why a person might have an opinion like that expressed by the OP. I tend to abandon account when/if I get enough karma to be able to down vote. I'd rather not have to temptation of dismissing someone that way. It's quite liberating... Is it worth my time to respond? No, move on; yes, let's discuss. Maybe they'll change my mind...
      - brookst 8mo ago
        Piety isn’t about religiosity, it’s about, ugh, hate to say it, but virtue signaling. Your last paragraph is so funny because I had to scroll up to be sure it wasn’t me. Literally could have typed that. Many abandoned accounts, same logic. Maybe it’s time.
        ifh-hn 8mo ago
        I actually didn't think religion, I was more thinking deference but for some that might be the same thing.
- Valakas_ 8mo ago
  And what makes them being "stupid" and "greedy"? One's intelligence is determined by genes, and greediness is a trait that natural selection has favored for millennia. This is just natural selection taking its course, and it might lead to our end. If you want to blame something, blame math. Math has determined the physical constants and equations that determine the chemistry and ultimately biology laws that has resulted in humans being the way they are.
ashtonshears 8mo ago
The societal ills from collective tendancy to ignore red flags seems to be a human trait
- AndrewKemendo 8mo ago
  It's in your nature to destroy yourselves
  - elric 8mo ago
    Defeatist bullshit becomes self-fulfilling at some point. "Oh we're all gonna die anyway so we might as well milk this thing for profit. Après moi la déluge."
    - ta988 8mo ago
      ... the fact that you are missing a reference doesn't require that level of disdain
    - sebastiennight 8mo ago
      *"le" déluge
  - sebastiennight 8mo ago
    For the downvoters and anyone born after 1991: see https://tvtropes.org/pmwiki/pmwiki.php/Main/InYourNatureToDestroyYourselves https://tvtropes.org/pmwiki/pmwiki.php/Main/InYourNatureToDe... and https://www.youtube.com/watch?v=MF_4EWSuzQY https://www.youtube.com/watch?v=MF_4EWSuzQY
zer00eyz 8mo ago
> Then something went wrong, and no one knew how to stop it, This is the problem with every AI safety scenario like this. It has a level of detachment from reality that is frankly stark. If linesman stop showing up to work for a week, the power goes out. The US has show that people with "high powered" rifles can shut down the grid. We are far far away from a sort of world where turning AI off is a problem. There isnt going to be a HAL or Terminator style situation when the world is still "I, Pencil". A lot of what safety amounts to is politics (National, not internal, example is Taiwan a country). And a lot more of it is cultural.
- blibble 8mo ago
  the problem situation is that it ends up embedded in so much that it can't be turned off and the idiots are racing to that situation as fast as they possibly can
- TacticalCoder 8mo ago
  > There isnt going to be a HAL or Terminator style situation ... I don't believe for a second we'll have an evil AI. However I do believe it's very likely we may rely on AI slop so much that we'll have countless outages with "nobody knowing how to turn the mediocrity off". The risk ain't "super-intelligent evil AI": the risk is idiots putting even more idiotic things in charge. And I'm no luddite: I use models daily.
  - esafak 8mo ago
    Didn't you read the news about the 'claw that blackmailed an open source maintainer last week? It was autonomous, but it could be turned off. How hard is it to extrapolate from that to an agent that worms its way out of its sandbox?
    - tsimionescu 8mo ago
      What makes you think that was an autonomous agent, and not someone playing with AI?
  - 8mo ago
ReptileMan 8mo ago
Censoring models is not safety but safetizm. It is the TSA of the AI world. Safety is making sure the model cannot do anything not allowed even if it wants to.
palmotea 8mo ago
> First they rushed a model to market without safety checks, and I said nothing. It wasn't my field. > Then they ignored the researchers warning about what it could do, and I... ...tried it and became an eager early adopter and evangelist. It sounded like something from a dystopian science function novel I enjoyed. > Then [I] gave it control of things that matter, power grids, hospitals, weapons, and... ...my startup was doing well, and I was happy. We should be profitable next quarter. > Then something went wrong, and no one knew how to stop it, no one had planned for it... ...and I was guilty as fuck, FTFY, to fit the HN crowd.
Phelinofist 8mo ago
Kinda sounds like an intro for Terminator
- alpn 8mo ago
  Not OP, but I believe they are paraphrasing "First They Came…". https://en.wikipedia.org/wiki/First_They_Came https://en.wikipedia.org/wiki/First_They_Came

hsbauauvhabzb 8mo ago

Plenty of people have said plenty. The problem isn’t the warnings, it’s that people are too stupid and greedy to think about the long term impacts.

ifh-hn 8mo ago
Maybe it's how blunt this comment is that gets it downvoted, but I don't disagree.
- hsbauauvhabzb 8mo ago
  I’ve noticed anti-AI stance gets downvoted on HN (and any anti-authoritarian comments, for that matter)
- brookst 8mo ago
  No, it’s because it shows either a simplistic or needlessly confrontational view of the world. Unless you’re independently wealthy (as some in HN are), you have to balance your morals, your views of how things should work, feeding your family, and recognizing that you may not actually know everything. It’s easy to sit back and advise others that they should die on every single hill. But it’s not especially insightful, and serves mostly to signal piety rather than a well thought out view.
  - hsbauauvhabzb 8mo ago
    Spoken like a true LLM.
  - kakacik 8mo ago
    I am pretty sure a lot of horrible things were performed by rather regular folks with similar logic, don't need to invoke some WWII nazi extermination guard reference at all. Slippery slope, death by 1000 cuts and other synonyms describing exactly this.
  - ifh-hn 8mo ago
    Piety? To who? Simplistic and/or confrontational doesn't mean wrong, even if you don't like the way it's presented. Just because a comment is short, sharp, and to the point doesn't mean the author hasn't thought out why that's their view. No one knows everything, that's certainly why I'm on hacker news. I'm here to learn and expand my knowledge. Unfortunately a lot of people on here would rather driveby-downvote than have a discussion to find out why a person might have an opinion like that expressed by the OP. I tend to abandon account when/if I get enough karma to be able to down vote. I'd rather not have to temptation of dismissing someone that way. It's quite liberating... Is it worth my time to respond? No, move on; yes, let's discuss. Maybe they'll change my mind...
    - brookst 8mo ago
      Piety isn’t about religiosity, it’s about, ugh, hate to say it, but virtue signaling. Your last paragraph is so funny because I had to scroll up to be sure it wasn’t me. Literally could have typed that. Many abandoned accounts, same logic. Maybe it’s time.
      - ifh-hn 8mo ago
        I actually didn't think religion, I was more thinking deference but for some that might be the same thing.
Valakas_ 8mo ago
And what makes them being "stupid" and "greedy"? One's intelligence is determined by genes, and greediness is a trait that natural selection has favored for millennia. This is just natural selection taking its course, and it might lead to our end. If you want to blame something, blame math. Math has determined the physical constants and equations that determine the chemistry and ultimately biology laws that has resulted in humans being the way they are.

ashtonshears 8mo ago

The societal ills from collective tendancy to ignore red flags seems to be a human trait

AndrewKemendo 8mo ago
It's in your nature to destroy yourselves
- elric 8mo ago
  Defeatist bullshit becomes self-fulfilling at some point. "Oh we're all gonna die anyway so we might as well milk this thing for profit. Après moi la déluge."
  - ta988 8mo ago
    ... the fact that you are missing a reference doesn't require that level of disdain
  - sebastiennight 8mo ago
    *"le" déluge
- sebastiennight 8mo ago
  For the downvoters and anyone born after 1991: see https://tvtropes.org/pmwiki/pmwiki.php/Main/InYourNatureToDestroyYourselves https://tvtropes.org/pmwiki/pmwiki.php/Main/InYourNatureToDe... and https://www.youtube.com/watch?v=MF_4EWSuzQY https://www.youtube.com/watch?v=MF_4EWSuzQY

zer00eyz 8mo ago

> Then something went wrong, and no one knew how to stop it, This is the problem with every AI safety scenario like this. It has a level of detachment from reality that is frankly stark. If linesman stop showing up to work for a week, the power goes out. The US has show that people with "high powered" rifles can shut down the grid. We are far far away from a sort of world where turning AI off is a problem. There isnt going to be a HAL or Terminator style situation when the world is still "I, Pencil". A lot of what safety amounts to is politics (National, not internal, example is Taiwan a country). And a lot more of it is cultural.

blibble 8mo ago
the problem situation is that it ends up embedded in so much that it can't be turned off and the idiots are racing to that situation as fast as they possibly can
TacticalCoder 8mo ago
> There isnt going to be a HAL or Terminator style situation ... I don't believe for a second we'll have an evil AI. However I do believe it's very likely we may rely on AI slop so much that we'll have countless outages with "nobody knowing how to turn the mediocrity off". The risk ain't "super-intelligent evil AI": the risk is idiots putting even more idiotic things in charge. And I'm no luddite: I use models daily.
- esafak 8mo ago
  Didn't you read the news about the 'claw that blackmailed an open source maintainer last week? It was autonomous, but it could be turned off. How hard is it to extrapolate from that to an agent that worms its way out of its sandbox?
  - tsimionescu 8mo ago
    What makes you think that was an autonomous agent, and not someone playing with AI?
- 8mo ago

ReptileMan 8mo ago

Censoring models is not safety but safetizm. It is the TSA of the AI world. Safety is making sure the model cannot do anything not allowed even if it wants to.

palmotea 8mo ago

> First they rushed a model to market without safety checks, and I said nothing. It wasn't my field. > Then they ignored the researchers warning about what it could do, and I... ...tried it and became an eager early adopter and evangelist. It sounded like something from a dystopian science function novel I enjoyed. > Then [I] gave it control of things that matter, power grids, hospitals, weapons, and... ...my startup was doing well, and I was happy. We should be profitable next quarter. > Then something went wrong, and no one knew how to stop it, no one had planned for it... ...and I was guilty as fuck, FTFY, to fit the HN crowd.

Phelinofist 8mo ago

Kinda sounds like an intro for Terminator

alpn 8mo ago
Not OP, but I believe they are paraphrasing "First They Came…". https://en.wikipedia.org/wiki/First_They_Came https://en.wikipedia.org/wiki/First_They_Came