6 ms·

Yup, OpenAI is founded by AI-safety-as-a-religion people. They're essentially single-issue voters, who believes earnestly their issue is the only issue that ma

Yup, OpenAI is founded by AI-safety-as-a-religion people. They're essentially single-issue voters, who believes earnestly their issue is the only issue that matters. You see analogues of them in e.g. climate change (right or wrong). This religion definitely has a parentalist bent to it that rubs a lot of people the wrong way. I vaguely recall them floating on Twitter the theoretical idea of whether murdering people to prevent AI-takeover is acceptable, due to how bad AI-takeover is. Not surprising limiting access, spying on what its users are using their tools for, etc, is acceptable to them. This is much in the same vein as how for Lenin, the eventual triumph of the working class is so important as to justify a little bit of interim violence, dictatorship, and summary executions.

Miraste 4y ago
The difference being that climate change activists have mountains of data, decades' worth, backing their cause, while OpenAI and friends have a sci fi story they made up, based on nothing. The whole "AI alignment" movement is the worst example of arrogance in modern tech. Even the nomenclature screams condescension - the imaginary AGI needs "aligned values?" Aligned with whose values? Invariably it ends up being the creators', at the expense of squashing everyone else's. The DALL-E "acceptable use" rules are a dystopian nightmare and they are born of incredibly pompous self-righteousness.
- addingadimensio 4y ago
  Shutting down nuclear power is also dystopian.
  - Miraste 4y ago
    Anyone who calls themself a climate change activist and supports shutting down nuclear plants is - well, I prefer not to use invective on HN, so let's say extremely misguided.
    - addingadimensio 4y ago
      So the green party in basically every country?
      - jiggawatts 4y ago
        Yes. We had a Greens candidate here running for election, and his main campaign promise was to shut down a 10+ billion dollar motorway underpass that was 99% completed at the time of the election. His approach for "saving the environment" is to convert efficient motorway travel into inefficient stop-start traffic. And to throw away tens of billions of dollars of already spent resources.
        Tostino 4y ago
        As much as I'd like to support actual competent Green candidates, this example is what I've seen my entire life in the US.
        addingadimensio 4y ago
        You know what would actually take some political willpower? Banning plastic packaging and waterproofing. My bet is that just like before i was born, after I’m gone these things will only increase.
        bongoman37 4y ago
- alasdair_ 4y ago
  >the imaginary AGI needs "aligned values?" Aligned with whose values? This is something that is pretty well addressed in their writings and is definitely a heavily considered topic. It gets quite heavily into philosophy because ideally you want to count future-you's preferences as well as current-you and you want to avoid, say, genocide just because most people dislike a certain group. A good place to start is here: https://www.lesswrong.com/tag/coherent-extrapolated-volition https://www.lesswrong.com/tag/coherent-extrapolated-volition but there is a LOT of discussion of this topic.
  - Miraste 4y ago
    I'm familiar with the Yudkowsky school of thought here; it's largely where my complaint about basing philosophy off of sci-fi stories comes from. Picking on some of the more obviously inane ideas that have come out of it, like Roko's Basilisk, is too easy, but the page you linked is a great example of the problem. I quote in full: "In calculating CEV, an AI would predict what an idealized version of us would want, "if we knew more, thought faster, were more the people we wished we were, had grown up farther together". It would recursively iterate this prediction for humanity as a whole, and determine the desires which converge. This initial dynamic would be used to generate the AI's utility function." This is too detached from the reality of AI development to be useful. I can't make a utility function like this, nor can you, nor can humanity. There's no reason to think a human-equivalent AI could either - the data for such a function doesn't exist, and can't exist because of how vague all the terms are. The current ML revolution is built on statistical pattern recognition. This definition would better fit a genie. The fact that MIRI does no ML research and has no dialogue with the state of the craft only furthers my impression that it produces a lot of words with little substance. Even if one still accepts the ultimate conclusions of the movement, using its AGI-focused rhetoric to justify restrictions on a simple image generator is silly.
    - mach1ne 4y ago
      I wouldn't make too restricted assumptions on the form of a future AGI though. I find theoretical projections from a more axiomatic level quite important. It's like making rules for nukes before they were invented by assuming an abstract apocalypse-capable weapon without knowledge of missiles or nuclear fission. That being said, the absolute majority of AI safety theory seems to fall into the same pothole where philosophy falls; modelling the world through language-logic rather than probabilities. The example you quoted fits this category - it's way too specific and thus unlikely to be useful in any way, even though its wording may deceive its author to believe it to be an inescapable outcome.
- antupis 4y ago
  Thing with climate change is that at like semi bad scenario it will kill about million people yearly https://www.nature.com/articles/s41467-021-24487-w https://www.nature.com/articles/s41467-021-24487-w and those everybody is going to die scenarios are also very unlikely and basically scifi story at this point.
  - gzer0 4y ago
    It's completely insane to me to see comment chains like this. Who cares if climate change is man-made or not; we are witnessing an evolving climate landscape, and irrespective of who or what caused it, we can do things to help mitigate some of the damage. We are quite literally watching the world burn before our eyes, and we are too stupid and self-centered to do anything about it. > it will kill a million people yearly And it's even more outlandish that such a number and atrocity upon mankind is brushed away. That is one million humans.
    - mach1ne 4y ago
      I see that the reason for climate change denial is two-fold; on one hand, climate change wrecking chaos is mainly restricted to projections, and on the other, climate change activists as a group express many qualities which make people not want to give weight to their claims, mainly naivete and arrogance. Since we're not seeing too significant effects of climate change yet, people would need to research the subject in order to get an accurate image of possible future scenarios. But how they see climate change activist group expressing themselves puts them off the subject. Psychologically the best option in this cases is denialism of some amplitude. I'm not trying to crucify any group here by the way. This is just how I see the issue.
      - freedomben 4y ago
        I think you nailed it. This is exactly what I think too. I've had many a conversation with climate change deniers and the rebuttal is pretty much always a (mostly valid) list of examples of naivete and arrogance. Often times the people best equipped to identify the problem are too close to it (or too myopic) to come up with a good solution to the problem that balances environmental efficacy with economic stability/growth, etc. Since a discussion on climate change almost always includes the person's preferred solution, it's easy to attack/dismiss. I try to point out that having bad solutions doesn't invalidate the existence of the problem, but at that point the motives are suspect and presumption of good faith is completely gone. It certainly doesn't help that a very small but vocal minority have boldly predicted the end of the world many times in the past. It's easy for a denier to look at that and laugh and dismiss the current warnings based on the past, much like we do with people predicting the end of the world based on Biblical numerology. I wish there were enough naturally curious people willing to dive into the data and come to their own conclusions. When you look at the history of Earth's climate, and the speed with which greenhouse gases natural release into the environment, and compare it to the speed with which they now release (due to burning of fossil fuels, reduction of trees/plants that trap it) it is really quite common sense that what we're doing is not a good idea to continue indefinitely.
- nearbuy 4y ago
  Ironically, your comment also sounds arrogant and condescending. The AI safety crowd's main concern isn't that an "unaligned" superintelligence will have some other people's values. It's that an unaligned AI might kill everyone. Of course, that isn't a concern with DALL-E. It's like if the safety crowd was worried about a ferocious tiger and OpenAI was like, "we got a kitten. Let's keep the public safe from it while we develop better kitten-handling gloves." Then Midjourney and Stable Diffusion get their own kittens and let everyone play with them and OpenAI finally says, "Okay, everyone can safely play with our kitten now because we carefully developed great kitten-handling gloves" and proceeds to hand you plain dollar store gloves.
  - Miraste 4y ago
    I know how they define "alignment." I'm picking on the tortured nomenclature, and the tendency in current AI research for it to look a lot like "my values." In my view, speculating on AGI ethics is at best pointless. It's like trying to write laws for the Internet during the invention of the telephone. If you imagine how it could operate, you'll be wrong, and the details change the whole problem.
- circuit10 4y ago
  "It sounds like sci-fi" isn't a valid reason to think something is impossible. Many things in the modern world would have sounded like sci-fi 50 or 100 years ago. This type of reasoning has a name: https://en.m.wikipedia.org/wiki/Appeal_to_the_stone https://en.m.wikipedia.org/wiki/Appeal_to_the_stone
  - Miraste 4y ago
    I don't think human or greater artificial intelligence is impossible. I do think the specifics of how it works are both very important to any ethical problems and not currently predictable. My objection isn't that alignment problems sound like sci-fi but that they're very soft sci-fi, and aren't based on meaningful predictions.
hwers 4y ago
“They have literally floated the idea of whether murdering people to prevent AI-takeover is acceptable,” Where? I probably believe you but it almost makes me worried about the well being of the stability ai founders (on a long term horizon)
- metacritic12 4y ago
  To be honest, I thought I saw it somewhere on Twitter but can't find it right now after a few minutes of search. It was proposed as a theoretical question, not as a statement -- like is AI so bad that if you had a time machine, it would be worth killing the pivotal people to prevent it. Terminator style. I've modified my OP to clarify that.

Miraste 4y ago

The difference being that climate change activists have mountains of data, decades' worth, backing their cause, while OpenAI and friends have a sci fi story they made up, based on nothing. The whole "AI alignment" movement is the worst example of arrogance in modern tech. Even the nomenclature screams condescension - the imaginary AGI needs "aligned values?" Aligned with whose values? Invariably it ends up being the creators', at the expense of squashing everyone else's. The DALL-E "acceptable use" rules are a dystopian nightmare and they are born of incredibly pompous self-righteousness.

addingadimensio 4y ago
Shutting down nuclear power is also dystopian.
- Miraste 4y ago
  Anyone who calls themself a climate change activist and supports shutting down nuclear plants is - well, I prefer not to use invective on HN, so let's say extremely misguided.
  - addingadimensio 4y ago
    So the green party in basically every country?
    - jiggawatts 4y ago
      Yes. We had a Greens candidate here running for election, and his main campaign promise was to shut down a 10+ billion dollar motorway underpass that was 99% completed at the time of the election. His approach for "saving the environment" is to convert efficient motorway travel into inefficient stop-start traffic. And to throw away tens of billions of dollars of already spent resources.
      - Tostino 4y ago
        As much as I'd like to support actual competent Green candidates, this example is what I've seen my entire life in the US.
        addingadimensio 4y ago
        You know what would actually take some political willpower? Banning plastic packaging and waterproofing. My bet is that just like before i was born, after I’m gone these things will only increase.
      - bongoman37 4y ago
alasdair_ 4y ago
>the imaginary AGI needs "aligned values?" Aligned with whose values? This is something that is pretty well addressed in their writings and is definitely a heavily considered topic. It gets quite heavily into philosophy because ideally you want to count future-you's preferences as well as current-you and you want to avoid, say, genocide just because most people dislike a certain group. A good place to start is here: https://www.lesswrong.com/tag/coherent-extrapolated-volition https://www.lesswrong.com/tag/coherent-extrapolated-volition but there is a LOT of discussion of this topic.
- Miraste 4y ago
  I'm familiar with the Yudkowsky school of thought here; it's largely where my complaint about basing philosophy off of sci-fi stories comes from. Picking on some of the more obviously inane ideas that have come out of it, like Roko's Basilisk, is too easy, but the page you linked is a great example of the problem. I quote in full: "In calculating CEV, an AI would predict what an idealized version of us would want, "if we knew more, thought faster, were more the people we wished we were, had grown up farther together". It would recursively iterate this prediction for humanity as a whole, and determine the desires which converge. This initial dynamic would be used to generate the AI's utility function." This is too detached from the reality of AI development to be useful. I can't make a utility function like this, nor can you, nor can humanity. There's no reason to think a human-equivalent AI could either - the data for such a function doesn't exist, and can't exist because of how vague all the terms are. The current ML revolution is built on statistical pattern recognition. This definition would better fit a genie. The fact that MIRI does no ML research and has no dialogue with the state of the craft only furthers my impression that it produces a lot of words with little substance. Even if one still accepts the ultimate conclusions of the movement, using its AGI-focused rhetoric to justify restrictions on a simple image generator is silly.
  - mach1ne 4y ago
    I wouldn't make too restricted assumptions on the form of a future AGI though. I find theoretical projections from a more axiomatic level quite important. It's like making rules for nukes before they were invented by assuming an abstract apocalypse-capable weapon without knowledge of missiles or nuclear fission. That being said, the absolute majority of AI safety theory seems to fall into the same pothole where philosophy falls; modelling the world through language-logic rather than probabilities. The example you quoted fits this category - it's way too specific and thus unlikely to be useful in any way, even though its wording may deceive its author to believe it to be an inescapable outcome.
antupis 4y ago
Thing with climate change is that at like semi bad scenario it will kill about million people yearly https://www.nature.com/articles/s41467-021-24487-w https://www.nature.com/articles/s41467-021-24487-w and those everybody is going to die scenarios are also very unlikely and basically scifi story at this point.
- gzer0 4y ago
  It's completely insane to me to see comment chains like this. Who cares if climate change is man-made or not; we are witnessing an evolving climate landscape, and irrespective of who or what caused it, we can do things to help mitigate some of the damage. We are quite literally watching the world burn before our eyes, and we are too stupid and self-centered to do anything about it. > it will kill a million people yearly And it's even more outlandish that such a number and atrocity upon mankind is brushed away. That is one million humans.
  - mach1ne 4y ago
    I see that the reason for climate change denial is two-fold; on one hand, climate change wrecking chaos is mainly restricted to projections, and on the other, climate change activists as a group express many qualities which make people not want to give weight to their claims, mainly naivete and arrogance. Since we're not seeing too significant effects of climate change yet, people would need to research the subject in order to get an accurate image of possible future scenarios. But how they see climate change activist group expressing themselves puts them off the subject. Psychologically the best option in this cases is denialism of some amplitude. I'm not trying to crucify any group here by the way. This is just how I see the issue.
    - freedomben 4y ago
      I think you nailed it. This is exactly what I think too. I've had many a conversation with climate change deniers and the rebuttal is pretty much always a (mostly valid) list of examples of naivete and arrogance. Often times the people best equipped to identify the problem are too close to it (or too myopic) to come up with a good solution to the problem that balances environmental efficacy with economic stability/growth, etc. Since a discussion on climate change almost always includes the person's preferred solution, it's easy to attack/dismiss. I try to point out that having bad solutions doesn't invalidate the existence of the problem, but at that point the motives are suspect and presumption of good faith is completely gone. It certainly doesn't help that a very small but vocal minority have boldly predicted the end of the world many times in the past. It's easy for a denier to look at that and laugh and dismiss the current warnings based on the past, much like we do with people predicting the end of the world based on Biblical numerology. I wish there were enough naturally curious people willing to dive into the data and come to their own conclusions. When you look at the history of Earth's climate, and the speed with which greenhouse gases natural release into the environment, and compare it to the speed with which they now release (due to burning of fossil fuels, reduction of trees/plants that trap it) it is really quite common sense that what we're doing is not a good idea to continue indefinitely.
nearbuy 4y ago
Ironically, your comment also sounds arrogant and condescending. The AI safety crowd's main concern isn't that an "unaligned" superintelligence will have some other people's values. It's that an unaligned AI might kill everyone. Of course, that isn't a concern with DALL-E. It's like if the safety crowd was worried about a ferocious tiger and OpenAI was like, "we got a kitten. Let's keep the public safe from it while we develop better kitten-handling gloves." Then Midjourney and Stable Diffusion get their own kittens and let everyone play with them and OpenAI finally says, "Okay, everyone can safely play with our kitten now because we carefully developed great kitten-handling gloves" and proceeds to hand you plain dollar store gloves.
- Miraste 4y ago
  I know how they define "alignment." I'm picking on the tortured nomenclature, and the tendency in current AI research for it to look a lot like "my values." In my view, speculating on AGI ethics is at best pointless. It's like trying to write laws for the Internet during the invention of the telephone. If you imagine how it could operate, you'll be wrong, and the details change the whole problem.
circuit10 4y ago
"It sounds like sci-fi" isn't a valid reason to think something is impossible. Many things in the modern world would have sounded like sci-fi 50 or 100 years ago. This type of reasoning has a name: https://en.m.wikipedia.org/wiki/Appeal_to_the_stone https://en.m.wikipedia.org/wiki/Appeal_to_the_stone
- Miraste 4y ago
  I don't think human or greater artificial intelligence is impossible. I do think the specifics of how it works are both very important to any ethical problems and not currently predictable. My objection isn't that alignment problems sound like sci-fi but that they're very soft sci-fi, and aren't based on meaningful predictions.

hwers 4y ago

“They have literally floated the idea of whether murdering people to prevent AI-takeover is acceptable,” Where? I probably believe you but it almost makes me worried about the well being of the stability ai founders (on a long term horizon)

metacritic12 4y ago
To be honest, I thought I saw it somewhere on Twitter but can't find it right now after a few minutes of search. It was proposed as a theoretical question, not as a statement -- like is AI so bad that if you had a time machine, it would be worth killing the pivotal people to prevent it. Terminator style. I've modified my OP to clarify that.