3 ms·

Maybe before saying you are going to align super human intelligence, you should first try to align merely human intelligence.

proc0 2y ago
Lol true and it got me thinking maybe AGI is what will finally align humanity together, whether its because it smart enough to fix the problem or dangerous enough it will require everyone's cooperation to stop it.
- labster 2y ago
  I doubt either will happen. Climate change was dangerous enough that it required everyone’s cooperation to stop it in the 1980s, and here we are still in denial. My only hope for the future is that multiple AGIs are just as dysfunctional as humans are.
  - kirth_gersen 2y ago
    Evil AI Overlord: "Silly humans! The only way to defeat me is to stop being bigoted and to transition to clean energy!" Humanity Collectively: "Oh no! Our only weakness!"
  - int_19h 2y ago
    If they are just as dysfunctional, they're going to start fighting wars against each other eventually, and will need cannon fodder for that just as human rulers do, no? ~
- gtirloni 2y ago
  > it will require everyone's cooperation to stop it Does this mythological AGI live in datacenters that we can shutdown or in some 5th dimension out of reach to us?
  - latentsea 2y ago
    It's a resilient, hyperscale distributed system. Just like humanity.
    - freehorse 2y ago
      Nobody has seen any AGI so far. As far as I am concerned, right now it is just a scarecrow used by sociopaths like Altman et al to promote their self-interests.
      - idiotsecant 2y ago
        Both things can be true. AGI is a boogeyman used by those with a competitive advantage to maintain that advantage artificially be casting competitor projects as dangerous and wreckless. AGI is a potential existential threat to the human species on the scale of global thermonuclear war and climate change.
      - latentsea 2y ago
        No one had ever seen nuclear weapons when they had only been theorized. I'm just assuming that since networked computers are everywhere, models seem to be proliferating and natural general intelligence seems to function as a resilient, distributed system, it's not unthinkable to me that things eventually trend in that direction.
        freehorse 2y ago
        Scientists had understanding of nuclear fission and how nuclear physics work. It was theoretically possible before it became practically feasible. We have no idea what intelligence really is ("resilient, distributed system" is not universally accepted as definition, even more to really mean something outside of analytic definitions if such have any use at all, to say the least), how it works, how a theoretical system could function as "generally intelligent" in concrete terms. Intelligence as we know it is a property of living organisms. It is a phenomenon. What "artificial intelligence" may even mean and if it is possible to create "intelligence" artificially is mere speculation. We face real existential threats that are not mere speculation, as we do not even know whether the ways we affect the planet may lead to a habitable environment for humans or not.
layer8 2y ago
Sam Altman works very hard at aligning human intelligence to his goals.
krisoft 2y ago
That is the main worry around the topic. The world is full of principal–agent problems. Politicians are supposed to serve the voters but they seem to be co-opted by their own self-interests and they go against the interests of the citizens. Burocracies and charitable organisations spend their energy to enlarge and entranche themselves instead of solving the problems they were created to solve for. CEOs optimise for their renumeration package instead of the long term benefits of the shareholders. Tradespeople cut corners where they know they won’t be caught before their payment is due. Police forces becomes corrupt and prefers to play statistical reporting games to look good instead of fighting crime. Our track record of aligning inteligences to goals is abysmal. We have a few tricks and we try them, but they are not that great. If we extrapolate this to artificial inteligences it is not difficult to foresee great catastropes.
deleted 2y ago
[deleted]

proc0 2y ago

Lol true and it got me thinking maybe AGI is what will finally align humanity together, whether its because it smart enough to fix the problem or dangerous enough it will require everyone's cooperation to stop it.

labster 2y ago
I doubt either will happen. Climate change was dangerous enough that it required everyone’s cooperation to stop it in the 1980s, and here we are still in denial. My only hope for the future is that multiple AGIs are just as dysfunctional as humans are.
- kirth_gersen 2y ago
  Evil AI Overlord: "Silly humans! The only way to defeat me is to stop being bigoted and to transition to clean energy!" Humanity Collectively: "Oh no! Our only weakness!"
- int_19h 2y ago
  If they are just as dysfunctional, they're going to start fighting wars against each other eventually, and will need cannon fodder for that just as human rulers do, no? ~
gtirloni 2y ago
> it will require everyone's cooperation to stop it Does this mythological AGI live in datacenters that we can shutdown or in some 5th dimension out of reach to us?
- latentsea 2y ago
  It's a resilient, hyperscale distributed system. Just like humanity.
  - freehorse 2y ago
    Nobody has seen any AGI so far. As far as I am concerned, right now it is just a scarecrow used by sociopaths like Altman et al to promote their self-interests.
    - idiotsecant 2y ago
      Both things can be true. AGI is a boogeyman used by those with a competitive advantage to maintain that advantage artificially be casting competitor projects as dangerous and wreckless. AGI is a potential existential threat to the human species on the scale of global thermonuclear war and climate change.
    - latentsea 2y ago
      No one had ever seen nuclear weapons when they had only been theorized. I'm just assuming that since networked computers are everywhere, models seem to be proliferating and natural general intelligence seems to function as a resilient, distributed system, it's not unthinkable to me that things eventually trend in that direction.
      - freehorse 2y ago
        Scientists had understanding of nuclear fission and how nuclear physics work. It was theoretically possible before it became practically feasible. We have no idea what intelligence really is ("resilient, distributed system" is not universally accepted as definition, even more to really mean something outside of analytic definitions if such have any use at all, to say the least), how it works, how a theoretical system could function as "generally intelligent" in concrete terms. Intelligence as we know it is a property of living organisms. It is a phenomenon. What "artificial intelligence" may even mean and if it is possible to create "intelligence" artificially is mere speculation. We face real existential threats that are not mere speculation, as we do not even know whether the ways we affect the planet may lead to a habitable environment for humans or not.

layer8 2y ago

Sam Altman works very hard at aligning human intelligence to his goals.

krisoft 2y ago

That is the main worry around the topic. The world is full of principal–agent problems. Politicians are supposed to serve the voters but they seem to be co-opted by their own self-interests and they go against the interests of the citizens. Burocracies and charitable organisations spend their energy to enlarge and entranche themselves instead of solving the problems they were created to solve for. CEOs optimise for their renumeration package instead of the long term benefits of the shareholders. Tradespeople cut corners where they know they won’t be caught before their payment is due. Police forces becomes corrupt and prefers to play statistical reporting games to look good instead of fighting crime. Our track record of aligning inteligences to goals is abysmal. We have a few tricks and we try them, but they are not that great. If we extrapolate this to artificial inteligences it is not difficult to foresee great catastropes.

deleted 2y ago

[deleted]