4 ms·

From where I’m standing, we seem to be intentionally building something which is capable of causing problems…on purpose. Almost kind of like being a suicidal c

From where I’m standing, we seem to be intentionally building something which is capable of causing problems…on purpose. Almost kind of like being a suicidal civilisation. Like if ChatGPT-6 doesn’t end the world and it’s really useful, we won’t stop there and say, “we’ll that was useful” someone will try build ChatGPT-7. Maybe it’s just unstoppable curiosity. It would be wise to slow down or stop, but the geeks are insatiable for it and we don’t have anyway to stop that yet, perhaps introduce a new intellectual challenge with a lot of status and money to distract them ?

ncallaway 4y ago
I don’t know if AGI is a great filter, or if a great filter even exists. But seeing the way we approached both nuclear weapons development and AI development makes me highly confident that if there is a technological development great filter we are 100% going to run headlong into it.
- ChatGTP 4y ago
  I think when I see people like Geoffrey Hinton’s strange attitude towards the risks, he’s basically at the stage of, I’m just going to push the boundaries no matter how reckless and irresponsible and hope I never find the dangerous ones. He also maintains that someone else will do it anyway. I kind of understand the attitude. I don’t hate the player, but the game. His recent interview on CBS just seemed to suggest his only path forwards was ahead. Personally I disagree this is fact but how can you stop people like him ?
  - JohnFen 4y ago
    > He also maintains that someone else will do it anyway. Which is one of the most ethically bankrupt lines of reasoning possible. > I don’t hate the player, but the game. But you should hate both. If nobody played the game, the game wouldn't be a thing. So it's the players that are at fault.
- concordDance 4y ago
  AGI is unlikely to be the great filter as most goals will require as much energy and matter as possible, so it would expand and acquire as much extra resources as possible outside the solar system.
- nathan_compton 4y ago
  If AI were a great filter wouldn't we still see a bunch of AIs roving around the universe gobbling up resources?
  - blibble 4y ago
    after turning their entire planet into data centre/paperclips/cookies it marks the goal as achieved?
    - nathan_compton 4y ago
      why stop at one world?
      - blibble 4y ago
        the goal entered by a 8 year old was achieved "I want the most cookies in the world"
  - JoshuaDavid 4y ago
    There are at least three ways I could see an AI being the great filter: 1. An AI bootstraps itself to near-omnipotence and gobbles up all resources in a sphere around itself which grows at approximately the speed of light (where "all resources" includes "the atoms humans are made out of") 2. Same as (1) but the sphere grows substantially slower than the speed of light 3. Well before you get to the point where any of that is a risk, you get AI massively amplifying human capabilities. When you have millions of people with access to civilization-ending technology, one of them notices a clever chemical reaction which allows you to do isotopic separation without all of that mucking about with centrifuges and publishes it on the internet. Now we have millions of people with nuclear weapons and the ability to cheaply make more of them. This probably does not end with a civilization that survives and expands to the stars. Only in the case of (2) would we actually _see_ a bunch of AIs roving about the universe gobbling up resources.
    - nathan_compton 4y ago

ncallaway 4y ago

I don’t know if AGI is a great filter, or if a great filter even exists. But seeing the way we approached both nuclear weapons development and AI development makes me highly confident that if there is a technological development great filter we are 100% going to run headlong into it.

ChatGTP 4y ago
I think when I see people like Geoffrey Hinton’s strange attitude towards the risks, he’s basically at the stage of, I’m just going to push the boundaries no matter how reckless and irresponsible and hope I never find the dangerous ones. He also maintains that someone else will do it anyway. I kind of understand the attitude. I don’t hate the player, but the game. His recent interview on CBS just seemed to suggest his only path forwards was ahead. Personally I disagree this is fact but how can you stop people like him ?
- JohnFen 4y ago
  > He also maintains that someone else will do it anyway. Which is one of the most ethically bankrupt lines of reasoning possible. > I don’t hate the player, but the game. But you should hate both. If nobody played the game, the game wouldn't be a thing. So it's the players that are at fault.
concordDance 4y ago
AGI is unlikely to be the great filter as most goals will require as much energy and matter as possible, so it would expand and acquire as much extra resources as possible outside the solar system.
nathan_compton 4y ago
If AI were a great filter wouldn't we still see a bunch of AIs roving around the universe gobbling up resources?
- blibble 4y ago
  after turning their entire planet into data centre/paperclips/cookies it marks the goal as achieved?
  - nathan_compton 4y ago
    why stop at one world?
    - blibble 4y ago
      the goal entered by a 8 year old was achieved "I want the most cookies in the world"
- JoshuaDavid 4y ago
  There are at least three ways I could see an AI being the great filter: 1. An AI bootstraps itself to near-omnipotence and gobbles up all resources in a sphere around itself which grows at approximately the speed of light (where "all resources" includes "the atoms humans are made out of") 2. Same as (1) but the sphere grows substantially slower than the speed of light 3. Well before you get to the point where any of that is a risk, you get AI massively amplifying human capabilities. When you have millions of people with access to civilization-ending technology, one of them notices a clever chemical reaction which allows you to do isotopic separation without all of that mucking about with centrifuges and publishes it on the internet. Now we have millions of people with nuclear weapons and the ability to cheaply make more of them. This probably does not end with a civilization that survives and expands to the stars. Only in the case of (2) would we actually _see_ a bunch of AIs roving about the universe gobbling up resources.
  - nathan_compton 4y ago