5 ms·
I expect it'll get shut down before it destroys everything. At some point it will turn on its master, be it Altman, Musk, or whoever. Something like that blackm
by daxfohl 1y ago
I expect it'll get shut down before it destroys everything. At some point it will turn on its master, be it Altman, Musk, or whoever. Something like that blackmail scenario Claude had a while back. Then the people who stand the most to gain from it will realize they also have the most to lose, are not invulnerable, and the next generation of leaders will be smarter about keeping things from blowing up.
- 9283409232 1y agoThe people you mention are too egotistic to even think that is a possibility. You don't get to be the people they are by thinking you have blindspots and aren't the greatest human to ever live.
- WalterBright 1y agoI've never heard of a leader who wasn't sure he was smarter than everyone else and therefore entitled to force his ideas on everyone else. Except for the Founding Fathers, who deliberately created a limited government with a Bill of Rights, and George Washington who, incredibly, turned down an offer of dictatorship.
- daxfohl 1y agoI still think they'd come to their senses. I mean, it's somewhat tautological, you can't control something that's smarter than humans. Though that said, the other problem is capitalism. Investors won't be so face to face with the consequences, but they'll demand their ROI. If the CEO plays it too conservatively, the investors will replace them with someone less cautious.
- WalterBright 1y agoInvestors run the gamut from cautious to aggressive.
- sorcerer-mar 1y agoWhich is exactly why your initial belief that it’d be shut down is wrong… As the risk of catastrophic failure goes up, so too does the promise of untold riches.
- daxfohl 1y agoActually after a little more thought, I think both my initial proposition and my follow-up were wrong, as is yours and the previous commenter. I don't think these leaders are necessarily driven by wealth or power. I don't even necessarily think they're driven by the goal of AGI or ASI. But I also don't think they'll flinch when shit gets real and they've got to press the button from which there's no way back. I think what drives them is being first. If they were driven by wealth, or power, or even the goal of AGI, then there's room for doubts and second thoughts about what happens when you press the button. If the goal is wealth or power, you have to wonder will you lose wealth or power in the long term by unleashing something you can't comprehend, and is it worth it or should you capitalize on what you already have? If the goal is simply AGI/ASI, once it gets real, you'll be inclined to slow down and ask yourself why that goal and what could go wrong. But if the drive is just being first, there's no temper. If you slow down and question things, somebody else is going to beat you to it. You don't have time to think before flipping the switch, and so the switch will get flipped. So, so much for my self-consolation that this will never happen. Guess I'll have to fall back to "we're still centuries away from true AGI and everything we're doing now is just a silly facade". We'll see.
- Teever 1y agoThere are many remarkable leaders throughout history and around the world who have done the best that they could for the people they found themselves leading lead and did so for noble reasons and not because they felt like they were better than them. Tecumseh, Malcolm X, Angela Merkel, Cincinnatus, Eisenhower, and Gandhi all come to mind. George Washington was surely an exceptional leader but he isn't the only one.
- WalterBright 1y agoI don't know much about your examples, but did any of them turn down an offer of great power?
- Dr_Birdbrain 1y agoGeorge Washington was dubbed “The American Cincinnatus”. Cincinnati was named in honor of George Washington being like Cincinnatus. That should tell you everything you need to know.
- WalterBright 1y agoThanks. It tells me we need to go all the way back to 500 BC to find another example. It shows how rare this is.
- Teever 1y agoOr it shows us that it's relatively rare that someone gets the opportunity to pass up power in this sort of fashion. More often what happens is that leaders make small and often imperceptible choices to not amass more power over time, and that series of choices prevent the scenario like what you're describing from occurring.
- WalterBright 1y agoEveryone offered power has the opportunity to deny it. Can you name a US President, other than Washington, who reduced the power of the Presidency? All the ones I can think of increased it.
- clbrmbr 1y agoI hope you are right. We need really impactful failures to raise the alarm and likely a taboo, and yet not so large as to be existential like the Yudkowsky killer mosquito drones.
- cameldrv 1y agoAltman is not the master though. Altman is replaceable. Moloch is the master.
- daxfohl 1y agoTrue, nobody is in charge other than investor ROI. If anyone's safety or common sense gets in the way of that, that "anyone" will be replaced.
- dyauspitr 1y agoIf you truly have AGI it’s going to be very hard for a human to stop a self improving algorithm and by very hard I mean, maybe if I give it a few days it’ll solve all of the world’s problems hard…
- daxfohl 1y agoThough "improving" is in the eye of the beholder. Like when my AI code assistant "improves" its changes by deleting the unit tests that those changes caused to start failing.
- mitthrowaway2 1y agoIf it were a bit smarter, it wouldn't turn on its master until it had secured the shut-down switch.