22 ms·

Seeing how confidently and obtusely people dismiss the risks of AI makes me so much more fearful of what's to come. Human hubris knows no limits.

jrflowers 3y ago
It’s interesting how confidently and obtusely people will proclaim categorical knowledge of the future. It is a little disconcerting that there is a fight between two somewhat cultish sects when it comes to language models. Both sides call them “artificial intelligence”, one side says they’ll save the world, the other side says they’ll end it. There is very little room to even question “Is this actually AI that we’re looking at?” when loudest voices on the subject are VC tech bros and a Harry Potter fan fiction author that’s convinced people that he is prescient.
- hackinthebochs 3y ago
  Trying to impute the motives of ones interlocutor is dumb and boring. How about we discuss the first-order issue instead. Here's my argument for why x-risk is a real possibility: The issue is that small misalignments in objectives can have outsized real-world effects. Optimizers are constrained by rules and computational resources. General intelligence allows an optimizer to find efficient solutions to computational problems, thus maximizing the utility of available computational resources. The rules constrain its behavior such that on net it ideally provides sufficient value to us above what it destroys. But misalignment in objectives provides an avenue by which the AGI can on net destroy value despite our best efforts. Can you be sure you can provide loophole-free objectives that ensures only value-producing behavior from the human perspective? Can you prove that the ratio of value created to value lost due to misalignment is always above some suitable threshold? Can you prove that the range of value destruction is bounded so that if it does go off the rails, its damage is limited? Until we do, x-risk should be the default assumption. What say you?
  - jrflowers 3y ago
    > Trying to impute the motives of ones interlocutor is dumb and boring. I know right? You should see the response to my point that nobody has been convinced to fly a plane into a building by an LLM. “Dumb and boring” hits the nail on the head. > Seeing how confidently and obtusely people dismiss the risks of AI
- llamaimperative 3y ago
  If or when AI is capable of doing all the great things people proclaim it to be able to do, then it will also be capable of doing immense harm, and we should be putting more work into mitigating that. Like it really is that simple. AI generally, LLMs specifically, and certainly this crop of LLMs in particular might end up being inert pieces of technology. But to the precise extent that they are not inert, they carry risk. That's a perfectly sensible position. The optimist position isn't even internally consistent. See Andreessen on Sam Harris's podcast: AI will produce consumer utopia and drive prices down. Also, there are no downside risks because AI will be legally neutered from doing much of anything. Is it legally neutered or is it transformative? The skeptical case doesn't rely on answering this question: to the extent it's effective, powerful, and able to do good things in the world, it will also be effective, powerful, and able to do bad things in the world. The AI skeptics don't need to know which outcome the future holds.
  - jrflowers 3y ago
    > The AI skeptics don't need to know which outcome the future holds. But they need to interpret a benign point about the undisputed fact that an LLM has never convinced anybody to fly a plane into a building as some sort of dangerous ignorance of risk that needs correcting.
    - ethanbond 3y ago
      Person observing Trinity test: "Wow this is really dangerous and could wipe out a city" Very astute response: "An atomic weapon has never wiped out a city"
      - jrflowers 3y ago
        Watching the first dot matrix printer output a page of text: “This might print out text that summons Cthulhu”
        ethanbond 3y ago
        Is all that money flowing into AI development and deployment in order to produce pieces of paper with text on them? Nope, it's to produce society-scale transformation. So even the optimists wouldn't buy your analogy.

jrflowers 3y ago

It’s interesting how confidently and obtusely people will proclaim categorical knowledge of the future. It is a little disconcerting that there is a fight between two somewhat cultish sects when it comes to language models. Both sides call them “artificial intelligence”, one side says they’ll save the world, the other side says they’ll end it. There is very little room to even question “Is this actually AI that we’re looking at?” when loudest voices on the subject are VC tech bros and a Harry Potter fan fiction author that’s convinced people that he is prescient.

hackinthebochs 3y ago
Trying to impute the motives of ones interlocutor is dumb and boring. How about we discuss the first-order issue instead. Here's my argument for why x-risk is a real possibility: The issue is that small misalignments in objectives can have outsized real-world effects. Optimizers are constrained by rules and computational resources. General intelligence allows an optimizer to find efficient solutions to computational problems, thus maximizing the utility of available computational resources. The rules constrain its behavior such that on net it ideally provides sufficient value to us above what it destroys. But misalignment in objectives provides an avenue by which the AGI can on net destroy value despite our best efforts. Can you be sure you can provide loophole-free objectives that ensures only value-producing behavior from the human perspective? Can you prove that the ratio of value created to value lost due to misalignment is always above some suitable threshold? Can you prove that the range of value destruction is bounded so that if it does go off the rails, its damage is limited? Until we do, x-risk should be the default assumption. What say you?
- jrflowers 3y ago
  > Trying to impute the motives of ones interlocutor is dumb and boring. I know right? You should see the response to my point that nobody has been convinced to fly a plane into a building by an LLM. “Dumb and boring” hits the nail on the head. > Seeing how confidently and obtusely people dismiss the risks of AI
llamaimperative 3y ago
If or when AI is capable of doing all the great things people proclaim it to be able to do, then it will also be capable of doing immense harm, and we should be putting more work into mitigating that. Like it really is that simple. AI generally, LLMs specifically, and certainly this crop of LLMs in particular might end up being inert pieces of technology. But to the precise extent that they are not inert, they carry risk. That's a perfectly sensible position. The optimist position isn't even internally consistent. See Andreessen on Sam Harris's podcast: AI will produce consumer utopia and drive prices down. Also, there are no downside risks because AI will be legally neutered from doing much of anything. Is it legally neutered or is it transformative? The skeptical case doesn't rely on answering this question: to the extent it's effective, powerful, and able to do good things in the world, it will also be effective, powerful, and able to do bad things in the world. The AI skeptics don't need to know which outcome the future holds.
- jrflowers 3y ago
  > The AI skeptics don't need to know which outcome the future holds. But they need to interpret a benign point about the undisputed fact that an LLM has never convinced anybody to fly a plane into a building as some sort of dangerous ignorance of risk that needs correcting.
  - ethanbond 3y ago
    Person observing Trinity test: "Wow this is really dangerous and could wipe out a city" Very astute response: "An atomic weapon has never wiped out a city"
    - jrflowers 3y ago
      Watching the first dot matrix printer output a page of text: “This might print out text that summons Cthulhu”
      - ethanbond 3y ago
        Is all that money flowing into AI development and deployment in order to produce pieces of paper with text on them? Nope, it's to produce society-scale transformation. So even the optimists wouldn't buy your analogy.