4 ms·

I've never heard of a leader who wasn't sure he was smarter than everyone else and therefore entitled to force his ideas on everyone else. Except for the Found

I've never heard of a leader who wasn't sure he was smarter than everyone else and therefore entitled to force his ideas on everyone else. Except for the Founding Fathers, who deliberately created a limited government with a Bill of Rights, and George Washington who, incredibly, turned down an offer of dictatorship.

daxfohl 1y ago
I still think they'd come to their senses. I mean, it's somewhat tautological, you can't control something that's smarter than humans. Though that said, the other problem is capitalism. Investors won't be so face to face with the consequences, but they'll demand their ROI. If the CEO plays it too conservatively, the investors will replace them with someone less cautious.
- WalterBright 1y ago
  Investors run the gamut from cautious to aggressive.
- sorcerer-mar 1y ago
  Which is exactly why your initial belief that it’d be shut down is wrong… As the risk of catastrophic failure goes up, so too does the promise of untold riches.
  - daxfohl 1y ago
    Actually after a little more thought, I think both my initial proposition and my follow-up were wrong, as is yours and the previous commenter. I don't think these leaders are necessarily driven by wealth or power. I don't even necessarily think they're driven by the goal of AGI or ASI. But I also don't think they'll flinch when shit gets real and they've got to press the button from which there's no way back. I think what drives them is being first. If they were driven by wealth, or power, or even the goal of AGI, then there's room for doubts and second thoughts about what happens when you press the button. If the goal is wealth or power, you have to wonder will you lose wealth or power in the long term by unleashing something you can't comprehend, and is it worth it or should you capitalize on what you already have? If the goal is simply AGI/ASI, once it gets real, you'll be inclined to slow down and ask yourself why that goal and what could go wrong. But if the drive is just being first, there's no temper. If you slow down and question things, somebody else is going to beat you to it. You don't have time to think before flipping the switch, and so the switch will get flipped. So, so much for my self-consolation that this will never happen. Guess I'll have to fall back to "we're still centuries away from true AGI and everything we're doing now is just a silly facade". We'll see.
Teever 1y ago
There are many remarkable leaders throughout history and around the world who have done the best that they could for the people they found themselves leading lead and did so for noble reasons and not because they felt like they were better than them. Tecumseh, Malcolm X, Angela Merkel, Cincinnatus, Eisenhower, and Gandhi all come to mind. George Washington was surely an exceptional leader but he isn't the only one.
- WalterBright 1y ago
  I don't know much about your examples, but did any of them turn down an offer of great power?
  - Dr_Birdbrain 1y ago
    George Washington was dubbed “The American Cincinnatus”. Cincinnati was named in honor of George Washington being like Cincinnatus. That should tell you everything you need to know.
    - WalterBright 1y ago
      Thanks. It tells me we need to go all the way back to 500 BC to find another example. It shows how rare this is.
      - Teever 1y ago
        Or it shows us that it's relatively rare that someone gets the opportunity to pass up power in this sort of fashion. More often what happens is that leaders make small and often imperceptible choices to not amass more power over time, and that series of choices prevent the scenario like what you're describing from occurring.
        WalterBright 1y ago
        Everyone offered power has the opportunity to deny it. Can you name a US President, other than Washington, who reduced the power of the Presidency? All the ones I can think of increased it.

daxfohl 1y ago

I still think they'd come to their senses. I mean, it's somewhat tautological, you can't control something that's smarter than humans. Though that said, the other problem is capitalism. Investors won't be so face to face with the consequences, but they'll demand their ROI. If the CEO plays it too conservatively, the investors will replace them with someone less cautious.

WalterBright 1y ago
Investors run the gamut from cautious to aggressive.
sorcerer-mar 1y ago
Which is exactly why your initial belief that it’d be shut down is wrong… As the risk of catastrophic failure goes up, so too does the promise of untold riches.
- daxfohl 1y ago
  Actually after a little more thought, I think both my initial proposition and my follow-up were wrong, as is yours and the previous commenter. I don't think these leaders are necessarily driven by wealth or power. I don't even necessarily think they're driven by the goal of AGI or ASI. But I also don't think they'll flinch when shit gets real and they've got to press the button from which there's no way back. I think what drives them is being first. If they were driven by wealth, or power, or even the goal of AGI, then there's room for doubts and second thoughts about what happens when you press the button. If the goal is wealth or power, you have to wonder will you lose wealth or power in the long term by unleashing something you can't comprehend, and is it worth it or should you capitalize on what you already have? If the goal is simply AGI/ASI, once it gets real, you'll be inclined to slow down and ask yourself why that goal and what could go wrong. But if the drive is just being first, there's no temper. If you slow down and question things, somebody else is going to beat you to it. You don't have time to think before flipping the switch, and so the switch will get flipped. So, so much for my self-consolation that this will never happen. Guess I'll have to fall back to "we're still centuries away from true AGI and everything we're doing now is just a silly facade". We'll see.

Teever 1y ago

There are many remarkable leaders throughout history and around the world who have done the best that they could for the people they found themselves leading lead and did so for noble reasons and not because they felt like they were better than them. Tecumseh, Malcolm X, Angela Merkel, Cincinnatus, Eisenhower, and Gandhi all come to mind. George Washington was surely an exceptional leader but he isn't the only one.

WalterBright 1y ago
I don't know much about your examples, but did any of them turn down an offer of great power?
- Dr_Birdbrain 1y ago
  George Washington was dubbed “The American Cincinnatus”. Cincinnati was named in honor of George Washington being like Cincinnatus. That should tell you everything you need to know.
  - WalterBright 1y ago
    Thanks. It tells me we need to go all the way back to 500 BC to find another example. It shows how rare this is.
    - Teever 1y ago
      Or it shows us that it's relatively rare that someone gets the opportunity to pass up power in this sort of fashion. More often what happens is that leaders make small and often imperceptible choices to not amass more power over time, and that series of choices prevent the scenario like what you're describing from occurring.
      - WalterBright 1y ago
        Everyone offered power has the opportunity to deny it. Can you name a US President, other than Washington, who reduced the power of the Presidency? All the ones I can think of increased it.