8 ms·

OpenAI is infected with AI safety brainworms

by namlem 4y ago

dmix 4y ago
AI ethics as a whole has become a bit of a joke. Any highly motivated group without much to do will seek out things to make themselves seem important and necessary.
- oefnak 4y ago
  I'm very interested why you think an advanced AI wouldn't be dangerous? Assume: - AGI wants to stay alive - Humans can create more AIs - Other AIs would compete for the same resources Then: Easiest way to make sure that they would get no more competitors would be...
  - ryanmcbride 4y ago
    AGI and what we're talking about here are completely different.
  - astrange 4y ago
    That’d be sad if we invented a superintelligent AI but still taught it the lump of labor fallacy. Anytime you see “creating an AI will obviously kill you” try reading it as “having children will obviously kill you” and see if it still makes sense.
    - alasdair_ 4y ago
      Most children aren't able to recursively improve their own hardware and software in short time spans, and generally most children are unlikely to be many orders of magnitude more intelligent than their parents.
      - astrange 4y ago
        Even in a world where an AI exists, why am I supposed to believe it's going to be able to do any of those things? I do believe it's possible to create one with the same attributes as a human person, it's just anything beyond that is unproven. Rather, it seems like evidence that singularitarianism is actually a religion (https://en.wikipedia.org/wiki/Millenarianism https://en.wikipedia.org/wiki/Millenarianism) which is why it believes things with magic powers will suddenly appear. In particular, exponential growth doesn't exist in nature and always turns into an S-curve… of course it's a problem if it doesn't level out until it's too late.
        cornel_io 4y ago
        What is your estimate of the probability that human intelligence is actually anywhere near the upper limit, rather than some point way further down the S-curve where seemingly exponential growth can still go for a long time? I'd bet a ton that we're nowhere near the top: evolution almost never comes up with the optimal solution for any problem, almost by definition it stops at "meh, good enough to reproduce". And you don't need a ton of intelligence to reproduce. Evolution's sub-optimality is actually one of the strongest arguments against intelligent design, so I'm really hesitant to agree that it requires any sort of leap to estimate that with some actual design it won't be very difficult to blow way past human intelligence once we can get there.
        astrange 4y ago
        > What is your estimate of the probability that human intelligence is actually anywhere near the upper limit, rather than some point way further down the S-curve where seemingly exponential growth can still go for a long time? Well, define "intelligence". People seem to use it in a vague way here - it might be what you call a motte and bailey. The motte (specific definition) is something like "can do math problems really fast" and the bailey is like "high executive function, is always right about everything, can predict the future". For the first one I don't think humans are near a limit, mostly because of the bottleneck in how we get born limiting our head sizes. But it is pretty good if you consider the costs of being alive - food requirements, heat dissipation, being bipedal, surviving being hit on the head, risk of brain cancer, etc, it's done well so far. Similarly an AI is going to have maintenance costs - the more RTX 3090s it runs on, the more calculations it might be able to do, but it's going to have to pay for them and their power bill, and they'll fail or give wrong answers eventually. And where's it getting the money anyway? As for the second kind I don't think you can be exponentially better at it than a human. At least if you are, it's not through intelligence, but it might be through access to more private information, or being rich enough to survive mistakes. As an example, you can't beat the stock market reliably with smarts, but you can by never being forced to sell. The real mystery to me is why people say "AI could recursively improve their own hardware and software in short time spans". I mean, that's clearly a made up concept since none of humans, computers or existing AI do it. But the closest thing I can think of is collective intelligence - humans individually haven't improved in the last 10k years, but we got a lot more humans and conquered everyone else that way. But we're also all individuals competing with each other and paying for our own individual food/maintenance/etc, which makes it different from nodes in an ever-growing AI.
    - hackerlight 4y ago
      That's disanalogous, because: - children won't be smarter than any living human. - children have a human brain which makes them predictable (constrained in behavior by current laws, institutions, and most likely a conscience). A better analogy is to ask what happens when a species branches off and evolves into a smarter species, but the dumber ancestor species still exists. I really don't understand the perspective that people in your position take. Is it that you don't think we'll arrive at super intelligent AI, and therefore there's little risk? Or that we will be able to control it? If you think we can control it, why? Like we're not that much smarter than our monkey ancestors and what hope did they ever have of stopping our absolute domination over them? And then all of you call this opinion a "religion" without even explaining why it's wrong?
      - mach1ne 4y ago
        Restricting AI to the lower end of human intelligence (e.g. around IQ of 70) makes it a useful resource which is guaranteed to be safe. A 70 IQ human couldn't take over the world nor disarm any safety features built into their bodies. You're right on the spot regarding the problem of having two different smart species on the same planet. We killed everything between us and chimps. Given enough time, the smarter species can be assumed to always take over.
        deleted 4y ago
        [deleted]
        astrange 4y ago
        That’s actually reverse causation - it’s not that we killed “everyone else”, it’s that everyone still alive became “us” after we gave up killing them and interbred with them instead. The British were going around genociding all over the place until pretty recently but now they’ve decided the Irish are human too. Homo sapiens is still expanding even: https://dna-explained.com/2012/11/16/the-new-root-haplogroup-a00/ https://dna-explained.com/2012/11/16/the-new-root-haplogroup...
        hackerlight 4y ago
  - namlem 4y ago
    AGI could be dangerous, but most people in AI safety are doing nothing useful to mitigate that risk.
- babyshake 4y ago
  My take on this is that AI ethics is really important, but just preventing AI from doing certain things like creating celebrity deepfakes is somewhat lazy and ineffective. A better application of AI ethics is developing technology that can reliably detect deepfakes, rather than just putting artificial limits in your product and acting like that is going to stop pandora's box from being opened.
  - marshray 4y ago
    If you develop technology that can reliably detect deepfakes (i.e., fn is_deepfake(m: media) -> bool), then couldn't an AI be trained to defeat it?
    - f1shy 4y ago
      I think it would lead to arms race.
    - waste_monk 4y ago
      Yes... but you can then train a model to detect the improved fakes. And then train a better deepfake model, and so on, forever. This technique of pitting two AI against each other (a generator and a detector/discriminator) is called a generative adversarial network, and it's used a lot for unsupervised training.
  - im3w1l 4y ago
    The funny thing about generated porn is that once it becomes ubiquitous real leaked tapes become deniable. So the possible downside for celebrities is that when they intentionally leak a tape to create buzz it may be met with a yawn.
    - ineedasername 4y ago
      These are still just images, not videos, so I don't understand the moral panic of "OMG celebrities & fake porn!" These systems aren't really lowering the barrier of entry on still-photography fake porn when previously anyone with Gimp & a few hours of video tutorials could churn out much the same thing. I think text-prompt generated deep fakes (not just of porn) will present a significantly larger challenge for society, but I don't see that same scope of problem on still images.
- pfisherman 4y ago
  I don’t think it is a joke, so much as misguided. I see a lot of focus on technical solutions, when the real problems are social. The big research question should not be “how can we build a ‘safe’ system” so much as “how should (or shouldn’t) we use these new tools and capabilities”?
- faeriechangling 4y ago
  “AI ethicists” so far smack of people who want to make AI express all their personal biases and prejudices. Listening to them for about 10 minutes makes me desperately hope for an open future like that created by stable diffusion out of the sheer terror they might have their hand on the rudder. I honestly think most of them are window dressing and aren’t allowed to have real influence though. They’re there for the PR, not to actually change things, but they honestly just make me really scared or big tech controlling AI.
biomcgary 4y ago
And, based on market share, they are clearly the target of a sophon directed by Roko's basilisk.
gedy 4y ago
I was enthusiastic about DALL-E but the "safety measures" are both heavy handed and naive. It gets in the way for many normal/reasonable prompts but seems easy to work around with various wordplay, so not sure the point. Stable Diffusion and others have been much easier to deal with.
- samatman 4y ago
  The harm is really hand-wavey and speculative, frankly. An image classifier calling Black faces gorillas? Embarrassing, insulting, has to be fixed. AI pre-crime classifiers for police departments? I'm against it, across the board. Do we really care that the image mulchers default to stereotypes? It means if you say "basketball player" they'll mostly be Black, if you just say "doctor" they'll mostly be white males (and probably balding with a stethoscope), but this can be qualified easily in the prompt. It just reflects the training data, and the smart thing to do is shrug and add enough words to get the image you want. It's not trying to throw shade, it literally understands nothing, it's not able to understand things, just match text prompts to generated images. Nerfing DALL-E by randomly adding 'diverse words' just makes it harder to dial in the image you want. Let's say you want a Vietnamese male doctor drinking coffee on break in Hanoi, it's not going to help you if 1/3rd of the images have "female" or "black" tagged onto it. It just seems low stakes. We wouldn't come after a human artist who happened to paint a picture which conforms to simple occupational stereotypes, why should AI be any different? It's not like it will refuse to give you what you want if you ask.
- ackfoobar 4y ago
  > heavy handed and naive It's good thing that the "safety measure" is the way it is - an afterthought. It means that those ideologues haven't yet had influence on the model itself.

OpenAI is infected with AI safety brainworms

by namlem 4y ago

OpenAI is infected with AI safety brainworms

dmix 4y ago

AI ethics as a whole has become a bit of a joke. Any highly motivated group without much to do will seek out things to make themselves seem important and necessary.

oefnak 4y ago
I'm very interested why you think an advanced AI wouldn't be dangerous? Assume: - AGI wants to stay alive - Humans can create more AIs - Other AIs would compete for the same resources Then: Easiest way to make sure that they would get no more competitors would be...
- ryanmcbride 4y ago
  AGI and what we're talking about here are completely different.
- astrange 4y ago
  That’d be sad if we invented a superintelligent AI but still taught it the lump of labor fallacy. Anytime you see “creating an AI will obviously kill you” try reading it as “having children will obviously kill you” and see if it still makes sense.
  - alasdair_ 4y ago
    Most children aren't able to recursively improve their own hardware and software in short time spans, and generally most children are unlikely to be many orders of magnitude more intelligent than their parents.
    - astrange 4y ago
      Even in a world where an AI exists, why am I supposed to believe it's going to be able to do any of those things? I do believe it's possible to create one with the same attributes as a human person, it's just anything beyond that is unproven. Rather, it seems like evidence that singularitarianism is actually a religion (https://en.wikipedia.org/wiki/Millenarianism https://en.wikipedia.org/wiki/Millenarianism) which is why it believes things with magic powers will suddenly appear. In particular, exponential growth doesn't exist in nature and always turns into an S-curve… of course it's a problem if it doesn't level out until it's too late.
      - cornel_io 4y ago
        What is your estimate of the probability that human intelligence is actually anywhere near the upper limit, rather than some point way further down the S-curve where seemingly exponential growth can still go for a long time? I'd bet a ton that we're nowhere near the top: evolution almost never comes up with the optimal solution for any problem, almost by definition it stops at "meh, good enough to reproduce". And you don't need a ton of intelligence to reproduce. Evolution's sub-optimality is actually one of the strongest arguments against intelligent design, so I'm really hesitant to agree that it requires any sort of leap to estimate that with some actual design it won't be very difficult to blow way past human intelligence once we can get there.
        astrange 4y ago
        > What is your estimate of the probability that human intelligence is actually anywhere near the upper limit, rather than some point way further down the S-curve where seemingly exponential growth can still go for a long time? Well, define "intelligence". People seem to use it in a vague way here - it might be what you call a motte and bailey. The motte (specific definition) is something like "can do math problems really fast" and the bailey is like "high executive function, is always right about everything, can predict the future". For the first one I don't think humans are near a limit, mostly because of the bottleneck in how we get born limiting our head sizes. But it is pretty good if you consider the costs of being alive - food requirements, heat dissipation, being bipedal, surviving being hit on the head, risk of brain cancer, etc, it's done well so far. Similarly an AI is going to have maintenance costs - the more RTX 3090s it runs on, the more calculations it might be able to do, but it's going to have to pay for them and their power bill, and they'll fail or give wrong answers eventually. And where's it getting the money anyway? As for the second kind I don't think you can be exponentially better at it than a human. At least if you are, it's not through intelligence, but it might be through access to more private information, or being rich enough to survive mistakes. As an example, you can't beat the stock market reliably with smarts, but you can by never being forced to sell. The real mystery to me is why people say "AI could recursively improve their own hardware and software in short time spans". I mean, that's clearly a made up concept since none of humans, computers or existing AI do it. But the closest thing I can think of is collective intelligence - humans individually haven't improved in the last 10k years, but we got a lot more humans and conquered everyone else that way. But we're also all individuals competing with each other and paying for our own individual food/maintenance/etc, which makes it different from nodes in an ever-growing AI.
  - hackerlight 4y ago
    That's disanalogous, because: - children won't be smarter than any living human. - children have a human brain which makes them predictable (constrained in behavior by current laws, institutions, and most likely a conscience). A better analogy is to ask what happens when a species branches off and evolves into a smarter species, but the dumber ancestor species still exists. I really don't understand the perspective that people in your position take. Is it that you don't think we'll arrive at super intelligent AI, and therefore there's little risk? Or that we will be able to control it? If you think we can control it, why? Like we're not that much smarter than our monkey ancestors and what hope did they ever have of stopping our absolute domination over them? And then all of you call this opinion a "religion" without even explaining why it's wrong?
    - mach1ne 4y ago
      Restricting AI to the lower end of human intelligence (e.g. around IQ of 70) makes it a useful resource which is guaranteed to be safe. A 70 IQ human couldn't take over the world nor disarm any safety features built into their bodies. You're right on the spot regarding the problem of having two different smart species on the same planet. We killed everything between us and chimps. Given enough time, the smarter species can be assumed to always take over.
      - deleted 4y ago
        [deleted]
      - astrange 4y ago
        That’s actually reverse causation - it’s not that we killed “everyone else”, it’s that everyone still alive became “us” after we gave up killing them and interbred with them instead. The British were going around genociding all over the place until pretty recently but now they’ve decided the Irish are human too. Homo sapiens is still expanding even: https://dna-explained.com/2012/11/16/the-new-root-haplogroup-a00/ https://dna-explained.com/2012/11/16/the-new-root-haplogroup...
        hackerlight 4y ago
- namlem 4y ago
  AGI could be dangerous, but most people in AI safety are doing nothing useful to mitigate that risk.
babyshake 4y ago
My take on this is that AI ethics is really important, but just preventing AI from doing certain things like creating celebrity deepfakes is somewhat lazy and ineffective. A better application of AI ethics is developing technology that can reliably detect deepfakes, rather than just putting artificial limits in your product and acting like that is going to stop pandora's box from being opened.
- marshray 4y ago
  If you develop technology that can reliably detect deepfakes (i.e., fn is_deepfake(m: media) -> bool), then couldn't an AI be trained to defeat it?
  - f1shy 4y ago
    I think it would lead to arms race.
  - waste_monk 4y ago
    Yes... but you can then train a model to detect the improved fakes. And then train a better deepfake model, and so on, forever. This technique of pitting two AI against each other (a generator and a detector/discriminator) is called a generative adversarial network, and it's used a lot for unsupervised training.
- im3w1l 4y ago
  The funny thing about generated porn is that once it becomes ubiquitous real leaked tapes become deniable. So the possible downside for celebrities is that when they intentionally leak a tape to create buzz it may be met with a yawn.
  - ineedasername 4y ago
    These are still just images, not videos, so I don't understand the moral panic of "OMG celebrities & fake porn!" These systems aren't really lowering the barrier of entry on still-photography fake porn when previously anyone with Gimp & a few hours of video tutorials could churn out much the same thing. I think text-prompt generated deep fakes (not just of porn) will present a significantly larger challenge for society, but I don't see that same scope of problem on still images.
pfisherman 4y ago
I don’t think it is a joke, so much as misguided. I see a lot of focus on technical solutions, when the real problems are social. The big research question should not be “how can we build a ‘safe’ system” so much as “how should (or shouldn’t) we use these new tools and capabilities”?
faeriechangling 4y ago
“AI ethicists” so far smack of people who want to make AI express all their personal biases and prejudices. Listening to them for about 10 minutes makes me desperately hope for an open future like that created by stable diffusion out of the sheer terror they might have their hand on the rudder. I honestly think most of them are window dressing and aren’t allowed to have real influence though. They’re there for the PR, not to actually change things, but they honestly just make me really scared or big tech controlling AI.

biomcgary 4y ago

And, based on market share, they are clearly the target of a sophon directed by Roko's basilisk.

gedy 4y ago

I was enthusiastic about DALL-E but the "safety measures" are both heavy handed and naive. It gets in the way for many normal/reasonable prompts but seems easy to work around with various wordplay, so not sure the point. Stable Diffusion and others have been much easier to deal with.

samatman 4y ago
The harm is really hand-wavey and speculative, frankly. An image classifier calling Black faces gorillas? Embarrassing, insulting, has to be fixed. AI pre-crime classifiers for police departments? I'm against it, across the board. Do we really care that the image mulchers default to stereotypes? It means if you say "basketball player" they'll mostly be Black, if you just say "doctor" they'll mostly be white males (and probably balding with a stethoscope), but this can be qualified easily in the prompt. It just reflects the training data, and the smart thing to do is shrug and add enough words to get the image you want. It's not trying to throw shade, it literally understands nothing, it's not able to understand things, just match text prompts to generated images. Nerfing DALL-E by randomly adding 'diverse words' just makes it harder to dial in the image you want. Let's say you want a Vietnamese male doctor drinking coffee on break in Hanoi, it's not going to help you if 1/3rd of the images have "female" or "black" tagged onto it. It just seems low stakes. We wouldn't come after a human artist who happened to paint a picture which conforms to simple occupational stereotypes, why should AI be any different? It's not like it will refuse to give you what you want if you ask.
ackfoobar 4y ago
> heavy handed and naive It's good thing that the "safety measure" is the way it is - an afterthought. It means that those ideologues haven't yet had influence on the model itself.