4 ms·

I don’t know if AGI is a great filter, or if a great filter even exists. But seeing the way we approached both nuclear weapons development and AI development m

I don’t know if AGI is a great filter, or if a great filter even exists. But seeing the way we approached both nuclear weapons development and AI development makes me highly confident that if there is a technological development great filter we are 100% going to run headlong into it.

ChatGTP 4y ago
I think when I see people like Geoffrey Hinton’s strange attitude towards the risks, he’s basically at the stage of, I’m just going to push the boundaries no matter how reckless and irresponsible and hope I never find the dangerous ones. He also maintains that someone else will do it anyway. I kind of understand the attitude. I don’t hate the player, but the game. His recent interview on CBS just seemed to suggest his only path forwards was ahead. Personally I disagree this is fact but how can you stop people like him ?
- JohnFen 4y ago
  > He also maintains that someone else will do it anyway. Which is one of the most ethically bankrupt lines of reasoning possible. > I don’t hate the player, but the game. But you should hate both. If nobody played the game, the game wouldn't be a thing. So it's the players that are at fault.
concordDance 4y ago
AGI is unlikely to be the great filter as most goals will require as much energy and matter as possible, so it would expand and acquire as much extra resources as possible outside the solar system.
nathan_compton 4y ago
If AI were a great filter wouldn't we still see a bunch of AIs roving around the universe gobbling up resources?
- blibble 4y ago
  after turning their entire planet into data centre/paperclips/cookies it marks the goal as achieved?
  - nathan_compton 4y ago
    why stop at one world?
    - blibble 4y ago
      the goal entered by a 8 year old was achieved "I want the most cookies in the world"
- JoshuaDavid 4y ago
  There are at least three ways I could see an AI being the great filter: 1. An AI bootstraps itself to near-omnipotence and gobbles up all resources in a sphere around itself which grows at approximately the speed of light (where "all resources" includes "the atoms humans are made out of") 2. Same as (1) but the sphere grows substantially slower than the speed of light 3. Well before you get to the point where any of that is a risk, you get AI massively amplifying human capabilities. When you have millions of people with access to civilization-ending technology, one of them notices a clever chemical reaction which allows you to do isotopic separation without all of that mucking about with centrifuges and publishes it on the internet. Now we have millions of people with nuclear weapons and the ability to cheaply make more of them. This probably does not end with a civilization that survives and expands to the stars. Only in the case of (2) would we actually _see_ a bunch of AIs roving about the universe gobbling up resources.
  - nathan_compton 4y ago
    Pretty sure we'd see them in case 1) and 2).
    - JoshuaDavid 4y ago

ChatGTP 4y ago

I think when I see people like Geoffrey Hinton’s strange attitude towards the risks, he’s basically at the stage of, I’m just going to push the boundaries no matter how reckless and irresponsible and hope I never find the dangerous ones. He also maintains that someone else will do it anyway. I kind of understand the attitude. I don’t hate the player, but the game. His recent interview on CBS just seemed to suggest his only path forwards was ahead. Personally I disagree this is fact but how can you stop people like him ?

JohnFen 4y ago
> He also maintains that someone else will do it anyway. Which is one of the most ethically bankrupt lines of reasoning possible. > I don’t hate the player, but the game. But you should hate both. If nobody played the game, the game wouldn't be a thing. So it's the players that are at fault.

concordDance 4y ago

AGI is unlikely to be the great filter as most goals will require as much energy and matter as possible, so it would expand and acquire as much extra resources as possible outside the solar system.

nathan_compton 4y ago

If AI were a great filter wouldn't we still see a bunch of AIs roving around the universe gobbling up resources?

blibble 4y ago
after turning their entire planet into data centre/paperclips/cookies it marks the goal as achieved?
- nathan_compton 4y ago
  why stop at one world?
  - blibble 4y ago
    the goal entered by a 8 year old was achieved "I want the most cookies in the world"
JoshuaDavid 4y ago
There are at least three ways I could see an AI being the great filter: 1. An AI bootstraps itself to near-omnipotence and gobbles up all resources in a sphere around itself which grows at approximately the speed of light (where "all resources" includes "the atoms humans are made out of") 2. Same as (1) but the sphere grows substantially slower than the speed of light 3. Well before you get to the point where any of that is a risk, you get AI massively amplifying human capabilities. When you have millions of people with access to civilization-ending technology, one of them notices a clever chemical reaction which allows you to do isotopic separation without all of that mucking about with centrifuges and publishes it on the internet. Now we have millions of people with nuclear weapons and the ability to cheaply make more of them. This probably does not end with a civilization that survives and expands to the stars. Only in the case of (2) would we actually _see_ a bunch of AIs roving about the universe gobbling up resources.
- nathan_compton 4y ago
  Pretty sure we'd see them in case 1) and 2).
  - JoshuaDavid 4y ago