3 ms·

> Our societies are not in any way equipped to deal with putting what may well be a sizable majority of working-age people out of work Our societies were built

> Our societies are not in any way equipped to deal with putting what may well be a sizable majority of working-age people out of work Our societies were built before there was a technology that could replace its smartest people with machines that never tire? I don't get why people keep trying to imagine current society + super-intelligent AI: by definition it won't be our current society if we can actually get there would it? I mean if we have AI that can even replace the researchers (I wouldn't dream so boldly tbh), imagine how much faster the pace of scientific discovery becomes. Imagine how much more efficient we can make power generation and transmission, discover new treatments for disease, democratize learning at costs never before possible... I don't love to spend too much time daydreaming what we could do down that because SWEs already feels like a bit of a pipedream, so all novel research being automated away is just completely in fantasy land... but realistically we're already on a pretty terrible trajectory otherwise. Our next billion people are about to be born into some of the worst off parts of the planet. AI becoming good enough to replace researchers would be an infinitely more positive trajectory than some of the others we could end up on on otherwise.

dividedbyzero 2y ago
Social media could have been utopian, too, yet those apps are algorithmic manipulation hellscapes that threaten to bring down even the most robust democracies. The same people who make it so are poised to be the ones in control of these AIs. I don't think they want the kind of utopia you imagine.
- BoorishBears 2y ago
  What I described doesn't have to be utopian in an absolute sense, just significantly better than where we're currently headed. I think a lot of the unchecked pessimism around super-intelligent AI is just people being a bit naive or shut off from the reality of just how terrible things are going to be over the next century. We're waging 25% tariffs over planefuls of people, what's going to happen when it's 100 million people trampling over borders trying to escape disease, famine, and temperatures incompatible with human life? Compared to that, even if these companies abuse their ownership of AI and monopolize the gains, an AI capable of producing novel research and development by itself would still bring us much closer to solving major problems than otherwise.
  - cookiengineer 2y ago
    I agree in part with your views. Though I think you misunderstand or underestimate human nature of self interest. Everyone that is in control of a superpower like this will abuse it. Be it on a presidential level, be it on CEO level, be it a major shareholder of a foreign NGO. That is why we had democratic splits of types of power in the first place. I say "had" because the trend globally is leading to right wing autocratic ideas due to manipulation of social media. Human self interest and egocentric world views is what gave us this mess. The only thing capable of evening out the odds is a federalistic decentralized approach, which we desperately need for AI. Something like a legislative system for lots of overfitted mini AI assistants that also give outliers a chance to be the social trend. Otherwise we will land up with the ministry of truth, which, right now is Facebook and TikTok effectively. The younger generations that grew up with social media tend heavily towards populist right wing ideas because those are easily marketable in 30 seconds. Paint the bad guy, say that it is established fact, next video. Nobody is interested in the rationale behind it, let alone finding and discussing a compromise like you would in a real debate that wants to find a solution. We need to find a way to change beliefs through rationale rather than emotions. Ironically this problem is also reflected in trained LLMs that turn into circlejerks because they've learned that from the dataset of us easily manipulateable humans.
    - c00kien1gg3r 2y ago
      [dead]
  - DaleMgrh 2y ago
    The problem emerges when the ones monopolizing the research use it for their own ends, which are to control everyone else. So instead of being used to improve quality of life, AI gets used to improve efficiency of death.
  - sensanaty 2y ago
    And you think the tools that are, as we speak, boiling rivers and lakes in order to power the insanely resource-hungry AIs is the solution to any of those problems, such as famine and rising temperatures? If anything, they're accelerating us towards these issues. There's about to be 500 billion dollars invested in generating even more electricity for these monstrosities, instead of literally anything else actually useful today that we could be putting towards climate research or renewable energy. Nope, we're just gonna generate even more spam and bullshit while spinning up nuclear reactors to power it all.
conartist6 2y ago
I don't think it's that simple. For one thing, AI isn't a democratizing force. If it's as good as you think it will be, it will be less like having a good education and more like having an indentured servant. Some people will have whole fleets of such servants doing their bidding, while others will have none. For another, research isn't an end unto itself. As you note, for some people an already-unfathomable level of societal knowledge has resulted in nothing but continued poverty. Benefit from scientific knowledge requires a stable economy full of consumers who can and will purchase high-tech items. Where will that wealth come from once the value of human intellectual labor has been so undercut by cheap AI intellectual labor? Without the capitol to make AIs work for us, most if not many people will be left to live their life as servants to AIs so that those AIs are able to have autonomy in the real world.

dividedbyzero 2y ago

Social media could have been utopian, too, yet those apps are algorithmic manipulation hellscapes that threaten to bring down even the most robust democracies. The same people who make it so are poised to be the ones in control of these AIs. I don't think they want the kind of utopia you imagine.

BoorishBears 2y ago
What I described doesn't have to be utopian in an absolute sense, just significantly better than where we're currently headed. I think a lot of the unchecked pessimism around super-intelligent AI is just people being a bit naive or shut off from the reality of just how terrible things are going to be over the next century. We're waging 25% tariffs over planefuls of people, what's going to happen when it's 100 million people trampling over borders trying to escape disease, famine, and temperatures incompatible with human life? Compared to that, even if these companies abuse their ownership of AI and monopolize the gains, an AI capable of producing novel research and development by itself would still bring us much closer to solving major problems than otherwise.
- cookiengineer 2y ago
  I agree in part with your views. Though I think you misunderstand or underestimate human nature of self interest. Everyone that is in control of a superpower like this will abuse it. Be it on a presidential level, be it on CEO level, be it a major shareholder of a foreign NGO. That is why we had democratic splits of types of power in the first place. I say "had" because the trend globally is leading to right wing autocratic ideas due to manipulation of social media. Human self interest and egocentric world views is what gave us this mess. The only thing capable of evening out the odds is a federalistic decentralized approach, which we desperately need for AI. Something like a legislative system for lots of overfitted mini AI assistants that also give outliers a chance to be the social trend. Otherwise we will land up with the ministry of truth, which, right now is Facebook and TikTok effectively. The younger generations that grew up with social media tend heavily towards populist right wing ideas because those are easily marketable in 30 seconds. Paint the bad guy, say that it is established fact, next video. Nobody is interested in the rationale behind it, let alone finding and discussing a compromise like you would in a real debate that wants to find a solution. We need to find a way to change beliefs through rationale rather than emotions. Ironically this problem is also reflected in trained LLMs that turn into circlejerks because they've learned that from the dataset of us easily manipulateable humans.
  - c00kien1gg3r 2y ago
    [dead]
- DaleMgrh 2y ago
  The problem emerges when the ones monopolizing the research use it for their own ends, which are to control everyone else. So instead of being used to improve quality of life, AI gets used to improve efficiency of death.
- sensanaty 2y ago
  And you think the tools that are, as we speak, boiling rivers and lakes in order to power the insanely resource-hungry AIs is the solution to any of those problems, such as famine and rising temperatures? If anything, they're accelerating us towards these issues. There's about to be 500 billion dollars invested in generating even more electricity for these monstrosities, instead of literally anything else actually useful today that we could be putting towards climate research or renewable energy. Nope, we're just gonna generate even more spam and bullshit while spinning up nuclear reactors to power it all.

conartist6 2y ago

I don't think it's that simple. For one thing, AI isn't a democratizing force. If it's as good as you think it will be, it will be less like having a good education and more like having an indentured servant. Some people will have whole fleets of such servants doing their bidding, while others will have none. For another, research isn't an end unto itself. As you note, for some people an already-unfathomable level of societal knowledge has resulted in nothing but continued poverty. Benefit from scientific knowledge requires a stable economy full of consumers who can and will purchase high-tech items. Where will that wealth come from once the value of human intellectual labor has been so undercut by cheap AI intellectual labor? Without the capitol to make AIs work for us, most if not many people will be left to live their life as servants to AIs so that those AIs are able to have autonomy in the real world.