3 ms·

> that's governments job Why do people take these AI "safety" research projects at face value? The real reason you need AI that is "safe" and "governable" is s

> that's governments job Why do people take these AI "safety" research projects at face value? The real reason you need AI that is "safe" and "governable" is so that when you start having it promote advertisers content or support the current administration, you don't have to worry about it going "off the rails" and promoting a competing product to criticizing the administration. I'm sure plenty of researchers in this space also believe they are working for the good of humanity, but I suspect the real am is much more practical and perfectly aligned with the business interests of all the companies sponsoring this type of work.

madeofpalk 9mo ago
Uh, I mean, I think we've seen first hand that generally one aspect of safety we want is to not produce sexualised images of children.
- js8 9mo ago
  Is it even an attainable goal? It seems an NN with less than say 4 billion parameters will be able to do that. The cost of training will likely go down with more models being available. Unless we lock down the computing for majority of people, I don't see how we can prevent someone creating a CSAM model in their garage. I don't want to see CSAM created, but the totalitarian control required is too much for my taste (and frankly it's preferable for that person to use NN than to go out and hurt actual children). Not to mention even locked down technology is often being abused by the privileged.
  - adrianN 9mo ago
    People can draw naked children with a stick and some dirt. I’m not sure that preventing the creation of fictional csam is the best use of our resources if we want to protect minors from abuse.
    - rainonmoon 9mo ago
      The best use? Probably not. But if I built a website that let people generate extremely convincing unlimited photos of you wearing an SS uniform and forcing your dog to smoke meth and sent them to everyone you’ve ever met, this might seem like a less worthy hill to die on. Or is that just a sticks and dirt thing too?
      - jorgenrodder 9mo ago
        Oddly specific... Did that really actually happen to you?
      - adrianN 9mo ago
        Everybody I care about would know that those pictures are not real so I think that the harm to me would likely be lower than the harm to society if building websites were impossible.
      - js8 9mo ago
        The people who are sending the pictures are criminally liable, regardless where they got them.The fact that somebody built a website for it is irrelevant, the act of sending them unsolicited is the immoral act here. (And frankly it's probably gonna be laughed off or end up as spam unless somebody you associate with is an idiot.)
  - madeofpalk 9mo ago
    The goal is not to prevent someone from making their own model do what they want, but to prevent your model from doing what you don't want, like generate CSAM or non-consentual sexually explicit photos. I don't understand how its controversial that someone/some-company might not want their products to be known for that.
    - js8 9mo ago
      It's a bit odd requirement, but.. OK. I mean who is the malicious actor here, the AI, the human user or the AI provider? If the AI, then we shouldn't give it an agency (user should always vet the output). If it's the user, the AI is irrelevant to the question. And if it's AI provider, why would they train AI on such materials in the first place. The whole enterprise of this kind of safety doesn't make much sense to me. If the AI is not able to follow so clear user instructions it's not ready for prime time and must be under human supervision at all times. (And subtle hallucinations on topic seem to be both more dangerous and more of a problem than blatant random production of explicit images, anyway.)
      - madeofpalk 9mo ago
        > It's a bit odd requirement It is a realistic requirement for dealing with user generated content at scale, because this is a realistic requirement thing some people will do. We’ve seen this fold out over the past month from Grok.
        noo_u 9mo ago
        [dead]
storus 9mo ago
We currently can't do "AI safety" even in bleeding edge alignment research so investing into some startups in that area is just burning money. Current LLMs/ViTs have non-zero probability of producing something unsafe and it's their inherent trait.
- zer00eyz 9mo ago
  > Current LLMs/ViTs have non-zero probability of producing something unsafe Show me on the LLM where the "unsafe" result is that doesn't already exist inside its training data set? Go ahead I will wait.
- soco 9mo ago
  Just so I understand your comment, are you trying to say that the money on safety should go first into research? And not into a later startup, which is just putting lipstick on the pig? I agree to this understanding, no doubt about it.
  - storus 9mo ago
    Basically yes. First figure out how to do reliable models that can be well-aligned with safety expectations, then invest into a startup that brings that tech to the industry.
al_borland 9mo ago
> promote advertisers content ... promoting a competing product Where is the value of AI when the responses are compromised like this? I could say the same thing about Google Search, which is one of the reasons I stopped using it. Are we betting on the masses not caring that they're being lied to for profit?

madeofpalk 9mo ago

Uh, I mean, I think we've seen first hand that generally one aspect of safety we want is to not produce sexualised images of children.

js8 9mo ago
Is it even an attainable goal? It seems an NN with less than say 4 billion parameters will be able to do that. The cost of training will likely go down with more models being available. Unless we lock down the computing for majority of people, I don't see how we can prevent someone creating a CSAM model in their garage. I don't want to see CSAM created, but the totalitarian control required is too much for my taste (and frankly it's preferable for that person to use NN than to go out and hurt actual children). Not to mention even locked down technology is often being abused by the privileged.
- adrianN 9mo ago
  People can draw naked children with a stick and some dirt. I’m not sure that preventing the creation of fictional csam is the best use of our resources if we want to protect minors from abuse.
  - rainonmoon 9mo ago
    The best use? Probably not. But if I built a website that let people generate extremely convincing unlimited photos of you wearing an SS uniform and forcing your dog to smoke meth and sent them to everyone you’ve ever met, this might seem like a less worthy hill to die on. Or is that just a sticks and dirt thing too?
    - jorgenrodder 9mo ago
      Oddly specific... Did that really actually happen to you?
    - adrianN 9mo ago
      Everybody I care about would know that those pictures are not real so I think that the harm to me would likely be lower than the harm to society if building websites were impossible.
    - js8 9mo ago
      The people who are sending the pictures are criminally liable, regardless where they got them.The fact that somebody built a website for it is irrelevant, the act of sending them unsolicited is the immoral act here. (And frankly it's probably gonna be laughed off or end up as spam unless somebody you associate with is an idiot.)
- madeofpalk 9mo ago
  The goal is not to prevent someone from making their own model do what they want, but to prevent your model from doing what you don't want, like generate CSAM or non-consentual sexually explicit photos. I don't understand how its controversial that someone/some-company might not want their products to be known for that.
  - js8 9mo ago
    It's a bit odd requirement, but.. OK. I mean who is the malicious actor here, the AI, the human user or the AI provider? If the AI, then we shouldn't give it an agency (user should always vet the output). If it's the user, the AI is irrelevant to the question. And if it's AI provider, why would they train AI on such materials in the first place. The whole enterprise of this kind of safety doesn't make much sense to me. If the AI is not able to follow so clear user instructions it's not ready for prime time and must be under human supervision at all times. (And subtle hallucinations on topic seem to be both more dangerous and more of a problem than blatant random production of explicit images, anyway.)
    - madeofpalk 9mo ago
      > It's a bit odd requirement It is a realistic requirement for dealing with user generated content at scale, because this is a realistic requirement thing some people will do. We’ve seen this fold out over the past month from Grok.
      - noo_u 9mo ago
        [dead]

storus 9mo ago

We currently can't do "AI safety" even in bleeding edge alignment research so investing into some startups in that area is just burning money. Current LLMs/ViTs have non-zero probability of producing something unsafe and it's their inherent trait.

zer00eyz 9mo ago
> Current LLMs/ViTs have non-zero probability of producing something unsafe Show me on the LLM where the "unsafe" result is that doesn't already exist inside its training data set? Go ahead I will wait.
soco 9mo ago
Just so I understand your comment, are you trying to say that the money on safety should go first into research? And not into a later startup, which is just putting lipstick on the pig? I agree to this understanding, no doubt about it.
- storus 9mo ago
  Basically yes. First figure out how to do reliable models that can be well-aligned with safety expectations, then invest into a startup that brings that tech to the industry.

al_borland 9mo ago

> promote advertisers content ... promoting a competing product Where is the value of AI when the responses are compromised like this? I could say the same thing about Google Search, which is one of the reasons I stopped using it. Are we betting on the masses not caring that they're being lied to for profit?