5 ms·

I know I'm an oddball when it comes to the stuff that crosses my mind, but here I go anyway. It's possible to stop developing things. It's not even hard; most

by dsign 2y ago

I know I'm an oddball when it comes to the stuff that crosses my mind, but here I go anyway. It's possible to stop developing things. It's not even hard; most of the world develops very little. Developing things requires capital, education, hard work, social stability and the rule of law. Many of us writing on this forum take those things for granted but it's more the exception than the rule, when you look at the entire planet. I think we will face the scenario of runaway AI, where we lose control, and we may not survive. I don't think it will be a sky-net type of thing, sudden. At least not at first. What will happen is that we will replace humans by AIs in more and more positions of influence and power, gradually. Our ChatGPTs of today will become board members and government advisors of tomorrow. It will take some decades--though probably not many. Then, a face-off will come one day, perhaps. Humans vs them. But if we do survive and come to regret the development of advanced AI and have a second chance, it will be trivially easy to suppress them: just destroy the semiconductor fabs, treat them the same way we treat ultra-centrifuges for enriching Uranium. Cut off the dangerous data centers, and forbid the reborn universities[1] from teaching linear algebra to the students. [1]: We will lose advanced education for the masses on the way, as it won't be economically viable nor necessary.

Simon_O_Rourke 2y ago
> What will happen is that we will replace humans by AIs in more and more positions of influence and power, gradually. Our ChatGPTs of today will become board members and government advisors of tomorrow. Great, can't wait for even some small improvement over the idiots in charge right now.
- moffkalast 2y ago
  I for one, also welcome our new Omnissiah overlords.
- realce 2y ago
  It's time to put an end to this fashionable and literal anti-human attitude. There's no comparative advantage to AI replacing humans en-masse because of how "stupid" we are. This POV is advocating for incalculable suffering and death. You personally will not be in a better or more rational position after this transition, you'll simply be dead.
  - Simon_O_Rourke 2y ago
    I don't see where you go from some over-hyped generative text bot outputting reams of semi-gibberish to.... AI will kill us all horribly. There's more than a few intractable technical limitations between ChatGPT and the T-1000.
deleted 2y ago
[deleted]
TheFuzzball 2y ago
I am so tired of the AI doomer argument. The entire thing is little more than a thought experiment. > Look at how fast AI has advanced, it you just project that trend out, we'll have human-level agents by the end of the decade. No. We won't. Scale up transformers as big as you like, this won't happen without massive advances in architecture and hardware. I believe it is possible, but the idea it'll happen any day now, and by accident is bullshit. This is one step from Pascal's Wager, but being presented as fact by otherwise smart people.
- dsign 2y ago
  > The entire thing is little more than a thought experiment. Yes. Nobody can predict the future. > but the idea it'll happen any day now, and by accident is bullshit. We agree on that one: it won't be sudden, and it won't be by accident. > I believe it is possible, but the idea it'll happen any day now, and by accident is bullshit. Exactly. Not by accident. But if you believe it's possible, then we are both doomers. The thing is, there are forces at play that want this. It's all of us. We in society want to remove other human beings from the chain of value. I use ChatGPT today to not pay a human editor. My boss uses Suno AI to play generated music with pro-productivty slogans before Teams meetings. The moment the owners of my enterprise believe it's possible to replace their highly paid engineers with AIs, they will do it. My bosses don't need to lift a finger today to ensure that future. Other people have already imagined it, and thus, already today we have well-founded AI companies doing their best to develop the technology. Their investors see an opportunity on making highly-skilled labor cheaper, and they are dumping their money into that enterprise. Better hardware, better models, better harnesses for those models. All of that is happening at speed. I'm not counting on accidents there. If anything, I'm counting on accidents Chernobyl style that make us realize, when there is still time, if we are stepping into danger.
philomath_mn 2y ago
> It's possible to stop developing things If the US were willing to compromise some of it's core values, then we could probably stop AI development domestically. But what about the rest of the world? If China or India want to reap the benefits of enhanced AI capability, how could we stop them? We can hit them with sanctions and other severe measures, but that hasn't stopped Russia in Ukraine -- plus the prospect of world-leading AI capability has a lot more economic value than what Ukraine can offer. So if we can't stop the world from developing these things, why hamstring ourselves and let our competitors have all of the benefits?
- hcurtiss 2y ago
  Exactly. Including military benefits. The US would not be a nation for long.
- hollerith 2y ago
  >the prospect of world-leading AI capability has a lot more economic value than what Ukraine can offer. The mere fact that you imagine that Moscow's motivation in invading Ukraine is economic is a sign that you're missing the main reasons Moscow or Beijing would want to ban AI: (1) unlike in the West and especially unlike the US, it is routine and normal for the government in those countries to ban things or discourage their use, especially new things that might cause large societal changes and (2) what Moscow and Beijing want most is not economic prosperity, but rather to prevent another one of those invasions or revolutions that kills millions of people and to prevent the country's ruling coalition from losing power.
  - philomath_mn 2y ago
    But this all comes back to the self-interest and game theory discussion. Let's suppose that, like you, both Moscow and Beijing do not want AGI to exist. What could they do about it? Why should they trust that the rest of the world will also pause their AI development? This whole discussion is basically a variation on the prisoner's dilemma. Either you cooperate and AI risks are mitigated, or you do not cooperate and try to take the best outcome for yourself. I think we can expect the latter. Not because it is the right thing or because it is the optimal decision for humanity, but because each individual will deem it their best choice, even after accounting for P(doom).
627467 2y ago
Everyone wants to be the prophet of doom of their own religion.
simonw 2y ago
"Our ChatGPTs of today will become board members and government advisors of tomorrow." That still feels like complete science fiction to me - more akin to appointing a complicated Excel spreadsheet as a board member.
- fritzo 2y ago
  It feels like mere language difference. Certainly every government official is advised by many Excel spreadsheets. Were those spreadsheets "appointed", no.
  - simonw 2y ago
    The difference is between AI tools as augmentation and AI tools as replacement. Board members using tools like ChatGPT or Excel as part of their deliberations? That's great. Replacing a board member entirely with a black box automation that makes meaningful decisions without human involvement? A catastrophically bad idea.
  - vladms 2y ago
    People like having someone to blame and fire and maybe send to jail. It's less impressive if someone blames everything on their Excel sheet...
    - deleted 2y ago
      [deleted]
jcarrano 2y ago
What if we face the scenario of a Dr. Manhattan type AGI, that's just fed up with people's problems and decides to leave us for the stars?
HelloMcFly 2y ago
This is my oddball thought: the thing about AI doomerism is that it feels to me like it requires substantially more assumptions and leaps of logic than environmental doomerism. And environmental doomerism seems only more justified as the rightward lurch of western societies continues. Note: I'm not quite a doomer, but definitely a pessimist.
anon291 2y ago
Right, let's go back to the stone age because we said so. > What will happen is that we will replace humans by AIs in more and more positions of influence and power, With all due respect, and not to be controversial, but how is this concern any more valid than the 'great replacement' worries.

by dsign 2y ago

Simon_O_Rourke 2y ago

> What will happen is that we will replace humans by AIs in more and more positions of influence and power, gradually. Our ChatGPTs of today will become board members and government advisors of tomorrow. Great, can't wait for even some small improvement over the idiots in charge right now.

moffkalast 2y ago
I for one, also welcome our new Omnissiah overlords.
realce 2y ago
It's time to put an end to this fashionable and literal anti-human attitude. There's no comparative advantage to AI replacing humans en-masse because of how "stupid" we are. This POV is advocating for incalculable suffering and death. You personally will not be in a better or more rational position after this transition, you'll simply be dead.
- Simon_O_Rourke 2y ago
  I don't see where you go from some over-hyped generative text bot outputting reams of semi-gibberish to.... AI will kill us all horribly. There's more than a few intractable technical limitations between ChatGPT and the T-1000.

deleted 2y ago

[deleted]

TheFuzzball 2y ago

I am so tired of the AI doomer argument. The entire thing is little more than a thought experiment. > Look at how fast AI has advanced, it you just project that trend out, we'll have human-level agents by the end of the decade. No. We won't. Scale up transformers as big as you like, this won't happen without massive advances in architecture and hardware. I believe it is possible, but the idea it'll happen any day now, and by accident is bullshit. This is one step from Pascal's Wager, but being presented as fact by otherwise smart people.

dsign 2y ago
> The entire thing is little more than a thought experiment. Yes. Nobody can predict the future. > but the idea it'll happen any day now, and by accident is bullshit. We agree on that one: it won't be sudden, and it won't be by accident. > I believe it is possible, but the idea it'll happen any day now, and by accident is bullshit. Exactly. Not by accident. But if you believe it's possible, then we are both doomers. The thing is, there are forces at play that want this. It's all of us. We in society want to remove other human beings from the chain of value. I use ChatGPT today to not pay a human editor. My boss uses Suno AI to play generated music with pro-productivty slogans before Teams meetings. The moment the owners of my enterprise believe it's possible to replace their highly paid engineers with AIs, they will do it. My bosses don't need to lift a finger today to ensure that future. Other people have already imagined it, and thus, already today we have well-founded AI companies doing their best to develop the technology. Their investors see an opportunity on making highly-skilled labor cheaper, and they are dumping their money into that enterprise. Better hardware, better models, better harnesses for those models. All of that is happening at speed. I'm not counting on accidents there. If anything, I'm counting on accidents Chernobyl style that make us realize, when there is still time, if we are stepping into danger.

philomath_mn 2y ago

> It's possible to stop developing things If the US were willing to compromise some of it's core values, then we could probably stop AI development domestically. But what about the rest of the world? If China or India want to reap the benefits of enhanced AI capability, how could we stop them? We can hit them with sanctions and other severe measures, but that hasn't stopped Russia in Ukraine -- plus the prospect of world-leading AI capability has a lot more economic value than what Ukraine can offer. So if we can't stop the world from developing these things, why hamstring ourselves and let our competitors have all of the benefits?

hcurtiss 2y ago
Exactly. Including military benefits. The US would not be a nation for long.
hollerith 2y ago
>the prospect of world-leading AI capability has a lot more economic value than what Ukraine can offer. The mere fact that you imagine that Moscow's motivation in invading Ukraine is economic is a sign that you're missing the main reasons Moscow or Beijing would want to ban AI: (1) unlike in the West and especially unlike the US, it is routine and normal for the government in those countries to ban things or discourage their use, especially new things that might cause large societal changes and (2) what Moscow and Beijing want most is not economic prosperity, but rather to prevent another one of those invasions or revolutions that kills millions of people and to prevent the country's ruling coalition from losing power.
- philomath_mn 2y ago
  But this all comes back to the self-interest and game theory discussion. Let's suppose that, like you, both Moscow and Beijing do not want AGI to exist. What could they do about it? Why should they trust that the rest of the world will also pause their AI development? This whole discussion is basically a variation on the prisoner's dilemma. Either you cooperate and AI risks are mitigated, or you do not cooperate and try to take the best outcome for yourself. I think we can expect the latter. Not because it is the right thing or because it is the optimal decision for humanity, but because each individual will deem it their best choice, even after accounting for P(doom).

627467 2y ago

Everyone wants to be the prophet of doom of their own religion.

simonw 2y ago

"Our ChatGPTs of today will become board members and government advisors of tomorrow." That still feels like complete science fiction to me - more akin to appointing a complicated Excel spreadsheet as a board member.

fritzo 2y ago
It feels like mere language difference. Certainly every government official is advised by many Excel spreadsheets. Were those spreadsheets "appointed", no.
- simonw 2y ago
  The difference is between AI tools as augmentation and AI tools as replacement. Board members using tools like ChatGPT or Excel as part of their deliberations? That's great. Replacing a board member entirely with a black box automation that makes meaningful decisions without human involvement? A catastrophically bad idea.
- vladms 2y ago
  People like having someone to blame and fire and maybe send to jail. It's less impressive if someone blames everything on their Excel sheet...
  - deleted 2y ago
    [deleted]

jcarrano 2y ago

What if we face the scenario of a Dr. Manhattan type AGI, that's just fed up with people's problems and decides to leave us for the stars?

HelloMcFly 2y ago

This is my oddball thought: the thing about AI doomerism is that it feels to me like it requires substantially more assumptions and leaps of logic than environmental doomerism. And environmental doomerism seems only more justified as the rightward lurch of western societies continues. Note: I'm not quite a doomer, but definitely a pessimist.

anon291 2y ago

Right, let's go back to the stone age because we said so. > What will happen is that we will replace humans by AIs in more and more positions of influence and power, With all due respect, and not to be controversial, but how is this concern any more valid than the 'great replacement' worries.