6 ms·

Ask HN: How did we go from wanting AGI to asking a 6 month pause on it?

• deleted 4y ago
  [deleted]
• duringmath 4y ago
  AGI isn't even imminent it's just the Twitter AI hype cycle has gone out of hand and is spilling over and producing this kind of silliness.
  • jlei523 4y ago
    I don't see it as hype to be honest. It's a real problem we need to solve now. Also, it seems like the letter was faked? Anyways, it doesn't matter if it's faked. It does make people notice that it's a real problem now. General consensus amongst data scientists of when each AI breakthrough will happen have almost always been wrong. AI breakthroughs tend to happen earlier than what the consensus predict. I don't see AGI predictions as any different.
    • sph 4y ago
      > It's a real problem we need to solve now. We need to solve AGI now? Don't get me wrong, it would be an incredible scientific breakthrough, but I cannot see how AGI would solve our current societal and human problems. In fact, it would just upturn our entire world. AGI is a tool we want, not the solution we need. And I agree with OP anyway. It is pure hype. Thinking we are closer to AGI because of LLM is just like thinking we are closer to the Moon because we have conquered the Everest peak.
      • jlei523 4y ago
        >We need to solve AGI now? No, we need to solve the problem of AI alignment, regulation, etc now. https://arxiv.org/pdf/2209.00626.pdf https://arxiv.org/pdf/2209.00626.pdf
• wmf 4y ago
  I haven't seen many people change position. The people who want to build AGI and the people who want to prevent AGI are different.
  • muzani 4y ago
    Actually, it seems like the people who are most worried about AGI are the ones building it, because they don't want to see it "raised" the wrong way. Here's a blog by Sam Altman, about 10 months before OpenAI was founded: https://blog.samaltman.com/the-software-revolution https://blog.samaltman.com/the-software-revolution "Two of the biggest risks I see emerging from the software revolution—AI and synthetic biology—may put tremendous capability to cause harm in the hands of small groups, or even individuals. It is probably already possible to design and produce a terrible disease in a small lab; development of an AI that could end human life may only require a few hundred people in an office building anywhere in the world, with no equipment other than laptops. ... I think the best strategy is to try to legislate sensible safeguards but work very hard to make sure the edge we get from technology on the good side is stronger than the edge that bad actors get. If we can synthesize new diseases, maybe we can synthesize vaccines. If we can make a bad AI, maybe we can make a good AI that stops the bad one."
    • giantg2 4y ago
      I don't really see how AI itself will end human life. It would have to interact with something else to achieve that.
      • muzani 4y ago
        You just answered it :p Microsoft trained AI on the internet and it became a racist asshole. OpenAI released ChatGPT and people had fun jailbreaking it to get it to do evil things. Someone could just put AI on a weaponless drone and it could start slashing kids at a playground, deliberately or not. I think what Sam Altman wanted to do was to make sure that if someone created an AI library that was as popular as MySQL, it wouldn't easily develop slasher drone behavior.
• p-e-w 4y ago
  Who is "we"? There are thousands of nuanced opinions on what AGI is and what it should be. So someone wrote a letter. That doesn't mean anything. It doesn't even mean that the average opinion has substantially changed. There is no consensus on anything here. Beware of so-called "experts" on this subject. There are no "AGI experts". There are AI experts, but speculation by humans on what a hypothetical superhuman intelligence might be like will never be more than, well, speculation. It's like ants trying to wrap their minds around what they imagine human philosophy to be. Your opinion on AGI is as good as mine, or as Elon Musk's, or as Eliezer Yudkowsky's. Nobody knows anything. Get comfortable with uncertainty, and don't jump to the conclusion that anything has happened just because somebody somewhere said something.
• TheAlchemist 4y ago
  One word - hype ! That being said, once we really reach the AGI level - I suppose it will be like this - very sudden.
• iExploder 4y ago
  Anyone with a bit of prescience would know well to follow the butlerian doctrine. In any way I think overhype is playing a role here as well. There is a lot of money to be made here, so motivations are high, tends to make people polarize their opinions.
• bimal2000 4y ago
  [flagged]
• mejutoco 4y ago
  Asking for a 6 months pause "to protect x" is just the excuse to further the interests of certain individuals.
• inphovore 4y ago
  These thoughts of a unified “we” are the first sign of self deception. We must all get along, yet there will not be found a universal consensus among humanity. Keep on going man, the future shall arrive, and not everyone will be ready. We must continue this debate of our human competence, relevance of public narratives, and true potential of outcomes. All the meanwhile the future presses upon us. Humans are still by far the greatest threat to our humanity.
  • muzani 4y ago
    "Humans are still by far the greatest threat to our humanity." This, but replace "humans" with "intelligence". One of the risks is augmented humans as well. A human with a chainsaw is more dangerous to other humans, even when that chainsaw is a tool. There's really no research done on what humans can actually do with AGI. Part of the fears are similar to industrial age. I think it's good though. We got through COVID from the lessons of past pandemics and we got through 2008 and 2023 from the lessons of past recessions and bubbles. We've already planned for AI not to be Skynet and Asimov has run several scenarios.
    • inphovore 4y ago
      I am shocked and mortified that you say “ We got through COVID from the lessons of past pandemics …” We collectively stumbled through incompetence, hysteria, delusion (on every side), and catastrophic failure in ways obscured by our ability to bs over the details.
      • qup 4y ago
        Which resulted in humanity's best response to a pandemic thus far.
        • GoldenMonkey 4y ago
          You do mean the pharmacy companies, most ingenious taking of profits… And authoritarians running roughshod over us all… regardless of effectiveness and outright harm by their ‘solutions’.
• chatmasta 4y ago
  Some authoritarians wrote a blog post.
• taf2 4y ago
  I believe the idea of a pause is ridiculous right now. AGI is the biggest thing since the iPhone in 2007 and Google in 98… anyone thinking about pausing is asking to be left in the dust . It’s go time like never before.
  • muzani 4y ago
    I've been saying it's more like the Watt steam engine. Which led to physical labor being cheap. Leading to railroads, cars, tanks, planes, drive bombers, battleships, tractors, the end of subsistence farming, people being forced to migrate to cities, child labor, 80-100 hour weeks, and so on. Maybe indirectly leading to things like the violent rise of communism, Cold War, and such. AI will lead to attention being cheap. AGI will be a wrecking ball to the education system. And it will mess with the creative industry who are suddenly realizing they were not that creative in the first place. I don't expect things to be as rosy as when Google and iPhone came in.
  • mikewarot 4y ago
    Think back in history to the interaction of the natives of the America with other humans, from Europe, with only slightly better technology. AGI would be make us the natives, only the disparity will grow geometrically. There is good reason to worry.
  • gnz11 4y ago
    AGI isn’t a thing yet.
• deleted 4y ago
  [deleted]
• Lionga 4y ago
  AGI (any reasonable definition of it) is not happening in our lifetime. We have a few very specialised "AI"s and still struggle since 30 years and trillions of USD investment to even just let an AI drive a car (in all conditions not just when everything is easy). The step from ChatGPT to an real AGI is much bigger then the step from going to the moon to colonising other planets that are lightyears away. Both AGI and colonising other planets lightyears away will not happen in our lifetime.
• giantg2 4y ago
  "We" might not have changed our ask at all. "We" might be referring to two divergent groups with minimal defectors. Many people have been urging caution for a long time. As we get closer to more features and capabilities those people are going to ride on the publicity of those achievements to call out their warnings. This will garner more interest because the threat seems closer or more real as the capabilities increase (or at least appear to be so due to company hype). Then there are many others who are excited and still pressing ahead.
• markus_zhang 4y ago
  If something may cause a hike of my chance of losing job I don't really care how rosy it sounds. Yes I use it very often but I prefer it to stay where it is. By the same principle I wish graphics tech to be fixed approximately in the level of late 90s. Well but I don't expect to get what I want. So this is basically different interest groups competing with each other. Usually the more technically advanced ones won towards the end but the more important thing is how long the other groups can drag it down. Under the current social-economic world order, any large scale tech advance such as AGI will almost surely bring destruction to a whole generation of workers. You think AGI, nuclear fusion and such will bring prosperity to us? Too naive, mon ami.