5 ms·

Yes, and specifically Yudkowsky, Bostrom, and MacAskill are “providing cover” for AI owners by prattling on about AI “waking up” and extinguishing humanity The

• kup0 3y ago
  I agree. I have plenty of concerns about AI, but those concerns involve real-world harms that can happen in the immediate/short-term, not just some hypothetical sci-fi mass casualty event and having AI criticisms focus on the wrong conversation really just compounds the short-term harm by shifting the conversation away Long before AI causes mass harm without human involvement, humans will find hundreds of ways to make it cause harm, and harm at scale. I do think the technology itself is part of the risk though, because of the flaws, the scale, etc inherent to its current iterations. However, maybe those are still the fault of humans for not giving it the proper limits, warnings, etc. to mitigate those things That said, it can be used for good in the right hands (accessibility tools, etc), potentially, though I'm certainly more of a doomer at this point in time.
• echelon 3y ago
  This is the real danger. What's worse, we've already seen this story play out once before. Why aren't we talking more about it?
• emtel 3y ago
  Please don't neglect to mention that a large number of very accomplished AI researchers share their concerns, or at the very least do not believe they should be dismissed out of hand.
  • deafpolygon 3y ago
    I'm reading that a good number of those accomplished researchers that work at these companies, that stand to gain if they ensure that their company remains at the helm of AI research. Better that than introduce competition and unpredictable results, possibly leaving them behind in a highly competitive industry with no moat.
    • kalkin 3y ago
      Are there messengers for the AI risk case who you'd consider credible, or is the message so outlandish in your mind that it doesn't really matter who's delivering it?
  • somesortofthing 3y ago
    Researchers aren't immune to the incentives the rest of the field has. They're every bit as financially incentivized to push the AI risk narrative as the rest of the industry.
    • acallaha 3y ago
      To name two Turing award winners: are you asserting that Geoff Hinton and Yoshua Bengio are pushing the AI risk narrative primarily because they're compromised by financial incentives? Geoff Hinton /left his job/ at Google because he sees the risks as real, so I think that's a tough case to make.
      • JumpCrisscross 3y ago
        > are you asserting that Geoff Hinton and Yoshua Bengio are pushing the AI risk narrative primarily because they're compromised by financial incentives Their message is being amplified and distorted by forces that are. That they're both comfortably wealthy from creating the problems they now preach solutions to is also no small matter.
      • worik 3y ago
        > Geoff Hinton /left his job/ at Google because he sees the risks as real, so I think that's a tough case to make. He did not leave his job when the bias and harm of the existing I models at Google were doing. He did not quit his job when Google fired their AI ethics champion rather than change their behaviour Has he had anything to say about the harm Google's algorithms do?
        • torginus 3y ago
          At least that's what he says. Maybe he got an unspecified offer to be the high priest of a to-be influential 'AI safety' organization. In my opinion, just like with the Patriot Act we are seeing a new world order trying to come into being. Come a decade, the few who bother and can learn about what transpires in the near future, will be shaking their heads at present humanity, asking how could we be so stupid and naive.
  • rurp 3y ago
    If they really believe that this is a significant risk then they are being incredibly reckless by not focusing all of their energy to halt AI development until a solution is found. Right now the opposite is happening, many researchers and companies who express these concerns are still barrelling ahead. A small group of people risking the lives of billions without their consent is morally repugnant.
    • kalkin 3y ago
      Many of the signatories of this statement did also sign the statement calling for a halt to big training runs earlier this year.
• circuit10 3y ago
  Can we not tackle both issues at the same time? But humanity being extinguished is clearly worse than some potential misuse of current AI so it makes sense to focus more on that
  • eropple 3y ago
    One of these things is real and happening now. It isn't potential. The other is a hypothetical future scenario that has literally no A->B->C line drawn to it, pushed by people whose ability to feed themselves depends on attention farming. So no. Let's not focus on the bullshit one, and let's focus on the one that is hurting people in the now.
    • circuit10 3y ago
      But there is a clear line of reasoning? AI has to be made with some kind of goal. Most goals don’t line up with humanity’s interests, and the AI will pursue them to the extreme as it’s designed for optimise aggressively for it. AI is easier to improve than biology, so once it gets good enough it will exponentially improve itself, as that allows it to achieve its goal better. Most solutions to this problem don’t actually work, we’ve yet to find a good one. It seems extremely short sighted to never consider any future situations that aren’t currently happening but seem likely to: https://xkcd.com/2278/ https://xkcd.com/2278/ There is a good real world example of how optimising for a goal aggressively can make the world unliveable for some species, which is humans destroying the environment for our economic growth. But at least there we do care about it and are trying to change it, because, among other reasons, we need the environment to survive ourselves. An AI wouldn’t necessarily need that, so it would treat us more like we treat bacteria or something, only keeping us around if necessary or if we don’t get in the way (and we will get in the way) It might sound silly but there is some solid reasoning behind it
      • eropple 3y ago
        Let me rephrase: a clear line of reasoning that isn't an Old Testament god with some Tron lines.
    • 3y ago
• Elyra 3y ago
  It's a distraction from the real problem by playing on people's fears in a way they can better understand. I see some parallels with how the fear of nuclear power was used as a distraction for the real issue of CO2 and fossil fuels.
• ben_w 3y ago
  AI don't need to wake up for a bug to kill y'all. While putting an AI in charge of weapons is at least three Hollywood plots[0], it has also (GOFAI is still AI) behind at least two real-life near-misses from triggering global thermonuclear war[1]. [0] Terminator (and the copycat called X-Men Days of Future Past); Colossus the Forbin Project; WarGames [1] Stanislav Petrov incident, 26 September 1983; Thule Site J incident October 5, 1960 (AKA "we forgot to tell it that the moon doesn't have an IFF transponder and that's fine")
• EVa5I7bHFq9mnYK 3y ago
  So true. I'd much prefer the AI to be used to solve captchas, or automatically navigate me to a live human in a tech support call, or filter spam without throwing every useful email into spam basket. Not gonna happen.
• kalkin 3y ago
  I'm curious if you see this argument as consistent with the narrative that the AI companies are just seeking regulatory capture. It seems to me they're inconsistent ways of understanding what is happening, since concern about misuse also seems to me to imply regulation. But I seem to see a lot of people on HN who say both things who give the impression that they're agreeing with each other.
  • jazzyjackson 3y ago
    They're in the business of renting GPU access - all the doomerism is marketing spend, oh this is going to be such advanced tech, you are going to be left behind, truly world changing stuff, rent your own GPU cluster today!
    • kalkin 3y ago
      That doesn't actually answer my question, and also seems a bit confused. OpenAI is not in the business of renting GPUs. And I've seen some Google DeepMind people talking about x-risk, but not the other cloud players (or Nvidia or TSMC).
      • jazzyjackson 3y ago
        > OpenAI is not in the business of renting GPUs. MS Azure Cloud is. Anthropic (very alignment/safety-centered company) is backed by Google and their cloud. Any API tokens sold are just cloud credits with markup. So to me the doomerism does come from those entities acting as fronts to the major players that would very much like to build a regulatory trench.
        • kalkin 3y ago
          If I have this right, you think academic signatories are being manipulated by OpenAI and Anthropic, who in turn are being manipulated by the major cloud players, all to sell GPUs? Setting aside the plausibility of this degree of coordination, I don't even see the alignment of interests. What does Google or Azure care of whether they sell GPUs to quasi-monopoly or a bunch of smaller competitors? In fact, isn't the standard economic argument that monopolies underproduce, so competition should if anything sell more GPUs? Meanwhile, promoting doomerism seems pretty risky for the GPU business - they have to promote it just enough to get hype and regulatory capture, but not enough that their ability to buy and use GPUs is actually restricted. Seems like a risky bet for a corporate exec to make, unless they can really control the doomers with puppet-like precision...
• NumberWangMan 3y ago
  Hiya Chubot, (fan of Oil by the way, thanks so much for your work!) I read the article you linked to, both parts. I wonder how much of people having psychotic breaks in the rationalist community is due to 1) people with tendencies toward mental illness gravitating toward rationalism or 2) rationalism being a way to avoid built-in biases in human thought, but those biases being important to keeping us sane on an individual level. (If you fully grasp the idea that everyone might die, and have an emotional reaction to that that's proportional compared to just one person you know, it can be devastating). I think we are bad at thinking about big numbers and risks, because being very good at evaluating risks is actually not great for short term, individual survival -- even if it's good for survival as a species. I know personally the whole AI/AGI thing has got me really down. It's a struggle to reconcile it with how little a lot of people seem to put stock in the idea of AGI ending up in control of humanity at some point. I totally agree that everything on your list is a real issue -- but I think that, even if we completely solve all those issues, how do we not end up with a society where most important decisions are eventually made by AGI? My assumptions are 1) that we eventually make AGI which is just superior to humans in terms of making decisions and planning, and 2) there will be significant pressure from capitalism and competition among governments to use AGI over people once that's the case. Similar to how automation has almost always won out over hand-production so far for manufactured goods. That's more the scenario Paul Christiano worries about than Yudkowsky. It seems more likely to me. But I still think that a lot of our mental heuristics about what we should worry about break down when it comes to the creation of something that out-guns us in terms of brainpower, and I think Yudkowsky makes a lot of good points about how we tend to shy away from facing reality when reality could be dreadful. That it's really easy to have a mental block about inventing something that makes humanity go extinct, where if it's possible to do that, there's no outside force that will swoop in and stop us like a parent stopping a child from falling into a river. If this is a real danger, we have to identify it in advance and take action to prevent it, even while there are a bunch of other problems to deal with, and even while a lot of people don't believe in it, and even while there's a lot of money to be made in the meantime but each bit takes us closer to a bad outcome. Reality isn't necessarily fair, we could be really screwed, and have all the problems you mentioned in addition to the risk of AGI killing us all (either right away or taking over and just gradually using up all the resources we need to live like we've done to so many species).