6 ms·

It's somewhat odd to me that many companies operating in the public eye are basically stating "We are creating a digital god, an instrument more powerful than a

• esafak 1y ago
  Lots of people in academia and industry are calling for more oversight. It's the US government that's behind. Europe's AI Act bans applications with unacceptable risk: https://en.wikipedia.org/wiki/Artificial_Intelligence_Act https://en.wikipedia.org/wiki/Artificial_Intelligence_Act
  • azinman2 1y ago
    Unless China handicaps the their progress as well (which they won’t, see made in China 2025), all you’re doing is handing the future to deepseek et al.
    • nicce 1y ago
      This thought process it not different than it was with nuclear weapons. The primary difference is the observability - with satellites we had some confidence that other nations respected treaties, or that they had enough reaction time for mutual destruction, but with this AI development we lack all that.
      • lukas099 1y ago
        Yes, it was the same with nukes, each side had to build them because the other side was building them.
      • bpodgursky 1y ago
        Only countries with nuclear weapons had an actual seat at the table when the world banned new nuclear weapon programs.
        • nicce 1y ago
          That is why we see the current AI competition and some attempts from companies to regulate it so that "it is safe only in their hands". https://time.com/6288245/openai-eu-lobbying-ai-act/ https://time.com/6288245/openai-eu-lobbying-ai-act/
    • esafak 1y ago
      What kind of a future is that? If China marches towards a dystopia, why should Europe dutifully follow? We can selectively ban uses without banning the technology wholesale; e.g., nuclear power generation is permitted, while nuclear weapons are strictly controlled.
      • alasano 1y ago
        We don't know whether pushing towards AGI is marching towards a dystopia. If it's winner takes all for the first company/nation to have AGI (presuming we can control it), then slowing down progress of any kind with regulation is a risk. I don't think there's a good enough analogy to be made, like your nuclear power/weapons example. The hypothetical benefits of an aligned AGI outweigh those of any other technology by orders of magnitude.
        • esafak 1y ago
          As with nuclear weapons, there is non-negligible probability of wiping out the human race. The companies developing AI have not solved the alignment problem, and OpenAI even dismantled what programs it had on it. They are not going to invest in it unless forced to. We should not be racing ahead because China is, but investing energy in alignment research and international agreements.
        • troupo 1y ago
          > We don't know whether pushing towards AGI is marching towards a dystopia. We do know that. By literally looking at China. > The hypothetical benefits of an aligned AGI outweigh those of any other technology by orders of magnitude. AGI aligned with whom?
      • BeetleB 1y ago
        > If China marches towards a dystopia, why should Europe dutifully follow? I think the more relevant question is: Do you want to live in a Chinese dystopia, or a European one?
        • esafak 1y ago
    • saubeidl 1y ago
      [dead]
    • immibis 1y ago
      Compare the other American "innovations" that Europe mostly rejects.
  • lenerdenator 1y ago
    The US government probably doesn't think it's behind. Right now it's operated by a bunch of people who think that you can directly relate the amount of money a venture could make in the next 90 days to its net benefit for society. Government telling them how they can and cannot make that money, in their minds, is government telling them that they cannot bring maximum benefit to society. Now, is this mindset myopic to everything that most people have in their lived experience? Is it ethically bankrupt and held by people who'd sell their own mothers for a penny if they otherwise couldn't get that penny? Would those people be banished to a place beyond human contact for the rest of their existence by functioning organs of an even somewhat-sane society? I don't know. I'm just asking questions.
    • aylmao 1y ago
      I'd go further and say the US government wants "an instrument more powerful than any nuclear weapon" to be built in its territory, by people it has jurisdiction over. It might not be a direct US-govt project like the Manhattan Project was, but it doesn't have to. The government has the ties it needs with the heads of all these AI companies, and if it comes to it, the US-govt has the muscle and legal authority to reign control over it. A good deal for everyone involved really. These companies get to make bank and technology that furthers their market dominance, the US-govt gets potentially "Manhattan project"-level pivotal technology— it's elites helping elites.
  • jimbokun 1y ago
    US government is behind because Biden admin were pushing strongly for controls and regulations and told Andersen and friends exactly that, who then went and did everything in their power to elect Trump, who then put those same tech bros in charge of making his AI policy.
  • philipwhiuk 1y ago
    > Lots of people in academia and industry Mostly OpenAI and DeepMind and it stunk of 'pulling up the drawbridge behind them' and pivoting from actual harm to theoretical harm. For a crowd supposedly entrenched in startups, it's amazing everyone here is so slow to recognise it's all funding pitches and contract bidding.
  • 0xDEAFBEAD 1y ago
    The EU can say all it wants about banning AI applications with unacceptable risk. But ASML is still selling machines to TSMC, which makes the chips which the AI companies are using. The EU is very much profiting off of the AI boom. ASML makes significantly more money than OpenAI, even.
    • kgwgk 1y ago
      If we think of “making money” as having more revenue than expenses a lemonade stand makes significantly more money than OpenAI.
• saubeidl 1y ago
  The EU does and has passed the AI act to reign in the worst consequences of this nuclear weapon. It has not been received well around here. The "digital god" angle might explain why. For many, this has become a religious movement, a savior for an otherwise doomed economic system.
  • rchaud 1y ago
    Absolutely. It's frankly quite shocking to see how otherwise atheist or agnostic people have so quickly begun worshipping at the altar of "inevitable AGI apocalypse", much in the same way as how extremist Christians await the rapture.
    • saubeidl 1y ago
      [dead]
    • lenerdenator 1y ago
      Roko's Basilisk is basically Pascal's wager with GPUs.
      • saubeidl 1y ago
        [dead]
    • Xenoamorphous 1y ago
      I guess they think that the “digital god” has a chance to become real (and soon, even), unlike the non-digital one?
      • rchaud 1y ago
        We'll be debating whether or not "AGI is here" in philosophical terms, in the same way people debate if God is real, for years to come. To say nothing of the untaxed "nonprofit" status these institutions share. Omnipotent deities can never be held responsible for famine and natural disasters ("God has a plan for us all"). AI currently has the same get-out-of-jail free card where mistakes that no literate human would ever make are handwaved away as "hallucinations" that can be exorcised with a more sophisticated training model ("prayers").
• otabdeveloper4 1y ago
  Well, because it's obviously bullshit and everyone knows it. Just play the game and get rich like everyone else.
  • esafak 1y ago
    Are you sure about that? AI-powered robotic soldiers are around the corner. What could go wrong...
    • devinprater 1y ago
      Ooo I know, Cybermen! Yay.
    • sealeck 1y ago
      Robot soldiers != AGI
      • deleted 1y ago
        [deleted]
    • otabdeveloper4 1y ago
      > AI agent robot soliders that are as inept as ChatGPT Sounds like payola for the enterprising and experienced mercenary.
• modeless 1y ago
  I don't know what sources you're reading. There's so much eye-batting I'm surprised people can see at all.
• atleastoptimal 1y ago
  Because many people fundamentally don’t believe AGI is possible at a basic level, even AI researchers. Humans tend to only understand what materially affects their existence.
• jimbokun 1y ago
  Most of us are batting our eyelashes as rapidly as possible but have no idea how to stop it.
• xandrius 1y ago
  How is an LLM more powerful than any nuclear weapon? Seriously curious.
  • kragen 1y ago
    Well, possibly it isn't. Possibly LLMs are limited in ways that humans aren't, and that's why the staggering advances from GPT-2 to GPT-3 and from GPT-3 to GPT-4 have not continued. Certainly GPT-4 doesn't seem to be more powerful than the largest nuclear weapons. But OpenAI isn't limited to creating LLMs. OpenAI's objective is not to create LLMs but to create artificial general intelligence that is better than humans at all intellectual tasks. Examples of such tasks include: 1. Designing nuclear weapons. 2. Designing and troubleshooting mining, materials processing, and energy production equipment. 3. Making money by investing in the stock market. 4. Discovering new physics and chemistry. 5. Designing and troubleshooting electronics such as GPUs. 6. Building better AI. 7. Cracking encryption. 8. Finding security flaws in computer software. 9. Understanding the published scientific literature. 10. Inferring unpublished discoveries of military significance from the published scientific literature. 11. Formulating military strategy. Presumably you can see that a system capable of doing all these things can easily be used to produce an unlimited quantity of nuclear weapons, thus making it more powerful than any nuclear weapon. If LLMs turn out not to be able to do those things better than humans, OpenAI will try other approaches, sooner or later. Maybe it'll turn out to be impossible, or much further off than expected, but that's not what OpenAI is claiming.
    • TrapLord_Rhodo 1y ago
      the problem is, none of that needs to happen. If the AI can start coming up with novel math or physics, it's game over. Whether the AI is "sentient" or not, being able to break that barrier would send us into an advancement spiral.
      • kragen 1y ago
        None of my argument depends on the AI being sentient. You are surely correct that there are weaker imaginable AIs than the strongly superhuman AI that OpenAI and I are talking about which would still be more powerful than nuclear weapons, but they are more debatable. For example, whether discovering new physics would permit the construction of new, more powerful weapons is debatable; it didn't help Archimedes or Tipu Sultan. So discussing such weak claims is likely to end up off in the weeds of logistics and speculation about exactly what kind of undiscovered physics and math would come to light. Instead, I focused on the most obviously correct ways that strongly superhuman AI would be more powerful than nuclear weapons. These may not be the most practically important ways. Maybe any strongly superhuman AI would immediately discover a way to explode the sun, or to control people's minds, or to build diamondoid molecular nanotechnology, or to genetically engineer super-plagues, or to collapse the false vacuum. Any of those would make nuclear weapons seem insignificant. But claims like those are much more uncertain than the very simple question before us: whether what OpenAI is trying to develop would be more powerful than nuclear weapons. Obviously it would be, by my reasoning in the grandparent comment, even if this isn't a false vacuum, if the sticky fingers problem makes diamondoid nanotechnology impossible, if people's minds are inherently uncontrollable, etc. So we don't need to resolve those other, more difficult questions in order to do the much easier task of ranking OpenAI's objective relative to nuclear weapons.
• squigz 1y ago
  We're all too busy rolling our eyes.
• gooob 1y ago
  have they started hiring people to make maglev trains and permaculture gardens all around urban areas yet?
• CorpOverreach 1y ago
  I'd really love to talk to someone that both really believes this to be true, and has a hands-on experience with building and using generative AI. The intersection of the two seems to be quite hard to find. At the state that we're in the AIs we're building are just really useful input/output devices that respond to a stimuli (e.g., a "prompt"). No stimuli, no output. This isn't a nuclear weapon. We're not going to accidentally create Skynet. The only thing it's going to go nuclear on is the market for jobs that are going to get automated in an economy that may not be ready for it. If anything, the "danger" here is that AGI is going to be a printing press. A cotton gin. A horseless carriage -- all at the same time and then some, into a world that may not be ready for it economically. Progress of technology should not be artitrarily held back to protect automateable jobs though. We need to adapt.
  • ev7 1y ago
    alignmentforum.com
  • voidspark 1y ago
    > Progress of technology should not be artitrarily held back to protect automateable jobs though. We need to adapt. So you don't mind if your economic value drops to zero, with all human labour replaced by machines? Dependent on UBI, existing in a basic pod, eating rations of slop.
    • cik 1y ago
      > So you don't mind if your economic value drops to zero, with all human labour replaced by machines? This was the fear when the cotton gin was invented. It was the ear when cars were created. The same complaint happened with the introduction of electronic, automated, telephone switchboards. Jobs change. Societies change. Unemployment worldwide, is near the lowest it has ever been. Work will change. Society will eventually move to a currency based on energy production, or something equally futuristic. This doesn't mean that getting there will be without pain.
      • voidspark 1y ago
• soheil 1y ago
  It'd be odd if people batted eyes before the 1st nuclear weapon came to be, but not batting now.