3 ms·

Are there any mechanisms right now that prevent a single actor from developing a human level intelligence or above? Think slightly better than GPT-4. What if t

Are there any mechanisms right now that prevent a single actor from developing a human level intelligence or above? Think slightly better than GPT-4. What if that actor is malicious, misguided or mistaken? What if that actor misaligns that intelligence accidentally?

constantcrying 3y ago
Then your chatbot produces bad output, that sucks.
- startupsfail 3y ago
  Imagine, you’ve never seen modern technology, say you are a kid in a very poor country. And then a helicopter flies into your village and starts shouting air-to-ground missiles. We might end up in a position of that kid, when “helicopters” built by such an AI will fly in. Don’t underestimate of what you don’t know and what is achievable with the right knowledge and technology. It might be a difference of a level of a villager and a level of technological civilization able to build a helicopter.
  - constantcrying 3y ago
    Seems very much like a science fiction story. I hate the whole AI debate, as it always seems to focus on some big "what if", but really the implications of AI running right now are already grave enough. Targeted advertisment/content are good enough to do effective oppinion control onto large parts of a population. But who cares about that if matrix multiplications could become sentient...
    - njanirudh 3y ago
      And do you know what 'effective opinion control' is used for? For controlling natural resources and destroying ideology that doesn't align with the flavour of the season.
      - constantcrying 3y ago
        I think it gets even better. You can target specific groups and push particular ideas to them. With that you can create an entire fake discourse. You probably don't even want that many people believing your particular ideology, they might demand standards and having enemies you control is about the best situation to be in.
        startupsfail 3y ago
        The end result would be that people will largely continue believing what they currently believe in. Nothing changes. Not risky.
    - startupsfail 3y ago
      The population had clearly been a subject of opinion control from various factions: from the church to TV to internet. And we’ve seen quite strong attempts before, including various nation level propaganda machines. All of these do work to some degree. They can be life threatening. But, unless these result in a nuclear war, these are not civilization-ending events. On the other hand, something slightly better (on a log scale) than GPT-4, can be a game changer.

constantcrying 3y ago

Then your chatbot produces bad output, that sucks.

startupsfail 3y ago
Imagine, you’ve never seen modern technology, say you are a kid in a very poor country. And then a helicopter flies into your village and starts shouting air-to-ground missiles. We might end up in a position of that kid, when “helicopters” built by such an AI will fly in. Don’t underestimate of what you don’t know and what is achievable with the right knowledge and technology. It might be a difference of a level of a villager and a level of technological civilization able to build a helicopter.
- constantcrying 3y ago
  Seems very much like a science fiction story. I hate the whole AI debate, as it always seems to focus on some big "what if", but really the implications of AI running right now are already grave enough. Targeted advertisment/content are good enough to do effective oppinion control onto large parts of a population. But who cares about that if matrix multiplications could become sentient...
  - njanirudh 3y ago
    And do you know what 'effective opinion control' is used for? For controlling natural resources and destroying ideology that doesn't align with the flavour of the season.
    - constantcrying 3y ago
      I think it gets even better. You can target specific groups and push particular ideas to them. With that you can create an entire fake discourse. You probably don't even want that many people believing your particular ideology, they might demand standards and having enemies you control is about the best situation to be in.
      - startupsfail 3y ago
        The end result would be that people will largely continue believing what they currently believe in. Nothing changes. Not risky.
  - startupsfail 3y ago
    The population had clearly been a subject of opinion control from various factions: from the church to TV to internet. And we’ve seen quite strong attempts before, including various nation level propaganda machines. All of these do work to some degree. They can be life threatening. But, unless these result in a nuclear war, these are not civilization-ending events. On the other hand, something slightly better (on a log scale) than GPT-4, can be a game changer.