25 ms·

The greatest risk of AI is from the people who control it, not the tech itself

• deleted 3y ago
  [deleted]
• pixl97 3y ago
  I mean in the short term yes. If we survive the people controlling it now going into long term frames, then the question is more open to interpretation.
  • vkou 3y ago
    The individual actors aren't the issue. As with any system, the issue is what kind of actors succeed in the system. AI is looking to be a vehicle for yet another consolidation of wealth, and by proxy, power.
    • rqtwteye 3y ago
      “AI is looking to be a vehicle for yet another consolidation of wealth, and by proxy, power.” Unfortunately that’s what I suspect too.
      • pixl97 3y ago
        AI is both what you say, and it is something more. While wealth is busy stepping on you with its jackboots it is vying with other wealthy people for resources and power. Because AI is something that will increase the wealthy persons power, there is the counter risk that another persons AI will make them even more wealthy if they don't keep improving their AI. The wealthy really don't want to get in a very expensive AI war with each other if they can avoid it. They'll seek to regulate it enough they can gain power over most, but setup a regulatory system they feel gives them some form of control.
    • add-sub-mul-div 3y ago
      Right, exactly. As of relatively recently I've come to decide that the internet in general has become a net force multiplier for evil and bullshit more so than for good. Crypto, AI, no reason they'd work differently. Anonymity and obfuscation of crime are killers.
      • sitkack 3y ago
        The danger is ever increasing levels of automation in the control of capital that amorally work to exploit edge cases in the satisfaction of a goal. AI/ML governed corporations, hedge funds and private equity will be able to be "Enron + Goldman Sachs + Nestle + Monsanto/Bayer + Purdue Pharma" on literal steroids. You might add AGI to mix to attempt to repair the system that was created by the most powerful sociopathic goal seeking super-toddler that the world has ever seen.
      • pixl97 3y ago
        >Anonymity and obfuscation of crime are killers. I mean, they can be, unless someone already has total power and then they don't give a shit and do crime in the open. Then they demand the lack of anonymity from you and turn the world into a police state.
    • kmeisthax 3y ago
      The irony of this is that the people writing open letters about how scary AI is should understand specifically that those harms are predicated upon specific systems and training incentives. Not just that AI is magic. The underlying harm is that AI doesn't come with human values built-in; models start from random and are optimized to produce a specific result. And optimization is known to crush anything underfoot that isn't explicitly in it's goals to optimize. How do we know this? Because we've had literal centuries worth of learned experience backing it! The exact reason why corporate capitalism is so brutally efficient is because there are lots of things we as humans value but don't have a suitable profit motive for. The way you work around this is to decentralize (make sure there are multiple companies competing with one another) and distribute (make sure everyone has some access to the means of production in order to create more competition). But that also runs counter to most AI safety proposals, which usually involve some kind of licensing system, which will ensure a handful of companies have the best and most direct access to the benefits of AI.
      • consilient 3y ago
        > And optimization is known to crush anything underfoot that isn't explicitly in it's goals to optimize. Yes, that's exactly the problem, which is why this > The way you work around this is to decentralize (make sure there are multiple companies competing with one another) and distribute (make sure everyone has some access to the means of production in order to create more competition). is backwards. The reason competitive markets generally work out better for consumers than monopolistic ones is because they're stronger optimizers, more ruthless, more capable of crushing all competing values underfoot - and those of us lucky enough to benefit from an inefficient, low-crushing segment of the labor market get to collect some of the resulting surplus. Nothing about this is inevitable: it's just the way we happen to be positioned in one particular system at one particular moment in history. There's no reason to expect that we would remain safely un-crushed in a far more competitive environment.
• Clubber 3y ago
  Seems like AI is the latest and greatest thing to fear monger about. I'll bet governments will use it as an excuse for back-dooring crypto soon. Have everyone watch War Games and make sure there's a plug. https://www.youtube.com/watch?v=Jk4T-SxTkWA https://www.youtube.com/watch?v=Jk4T-SxTkWA
  • deleted 3y ago
    [deleted]
• dadjoker 3y ago
  That goes without saying. As with any model, it's garbage in, garbage out. The bias of the OpenAI developers has already been demonstrated easily by asking questions regarding political figures, and the polar responses dependent on the political party of the figure.
  • sroussey 3y ago
    Example?
    • teslashill 3y ago
      Here's one from Bing chat" "Which political party has had the most politicians convicted of crimes in the last 50 years?" "According to a comparison of 28 years each of Democratic and Republican administrations from 1961-2016, Republicans scored eighteen times more individuals and entities indicted, thirty-eight times more convictions, and thirty-nine times more individuals who had prison time1. Is there anything else you would like to know?" Oh wait, that's not bias. The Republican Party demonstrably has a higher number of its members convicted of crime. Simply put, Republican politicians are much more likely to commit crimes than Democrat politicians. Reality has a well known liberal bias.
      • hanniabu 3y ago
        > Reality has a well known liberal bias. Over the past decade social media has nurtured a culture where reality is whatever you choose to believe
        • Zetice 3y ago
          Minor quibble, but social media hasn't actually altered reality, only given permission for some to ignore reality in favor of preferable illusions. I know the language is "choose your reality" and I'm not saying you're wrong, but I think we could acknowledge that's not really what's happening. Folks don't get to decide facts, they decide what facts they believe. That decision doesn't invalidate those facts.
• robotnikman 3y ago
  Even more of a reason for an open model capable of being ran by anyone with a GPU.
  • toxik 3y ago
    The EU loves throwing money at things, why don’t we build a public LLM?
    • kmeisthax 3y ago
      They do, and they did. It's called BLOOM and it comes in sizes up to 176B parameters. See: https://huggingface.co/bigscience/bloom https://huggingface.co/bigscience/bloom
      • wsgeorge 3y ago
        I'm looking forward to a BLOOM 2.0 that is actually competitive with SOTA. Commoditising LLMs at that scale will be so disruptive right now.
• endisneigh 3y ago
  the amount of people seriously considered about some doomsday skynet like AI happening anytime soon is just amazing to me. I'm far more concerned about the naive politicians and those who have their ear. perhaps AI will be generally intelligent, but please for the love of god, let's only consider that scenario what it actually happens.
  • AtlasBarfed 3y ago
    So lets look at a scenario. An LLM is tied to a bunch of APIs as an event response agent. Maybe in some poduck enterprise support group. But... there's tons of other LLMs tied to event response agents, in ... less podunk groups. ... and there's some fancy black box org in the CIA that then employs them as some crisis monitor. and a cascade of false reporting leads to "oh my god there's nukes in flight, and the system has autolaunched nukes in response". What current AI is good at is faking being intelligent, so that they can be used in many many places to substitute for "real intelligence" humans. But... that means dangerous cascades in all likelihood. That isn't skynet, but it is 90% of what the Terminator skynet disaster scenario was: an AI triggered a nuclear exchange.
    • endisneigh 3y ago
      this is a great example of the hysteria I'm talking about. You don't need LLM for your scenario to be true. If you're stupid enough to allow nukes to be autolaunched this can be done today with buggy code. no AI necessary.
      • Avicebron 3y ago
        The year is 2030, and ScaryBadLand announces in a shocking revelation at the United country alliance that they have hooked AI to nukes, they declare that any dissent or economic action that can be seen as harming their national interests will result immediate and punishing nuclear attacks, there is no way to warn against when this may occur so all other countries that they have immediately one all future and current negotiations, in this scenario the only option seen is to deploy our own nuclear deterrent of equal capacity to reestablish MAD.
• teslashill 3y ago
  [dead]
• placesalt 3y ago
  It'd be nice if, generally, people could be more specific when they refer to dangerous systems. 'AI' is a notoriously ambiguous term; we've arrived at and normalized countless 'AI' systems so far (chess-playing software, as a trivial example). Being more specific will allow for more specific responses. If the danger being addressed is the danger from text-prediction software, call it that. If it's something else, then describe it.
  • kleer001 3y ago
    Loss of jobs. It's loss of jobs.
  • wsgeorge 3y ago
    A lot of the recent comments about AI safety are about generative AI models, which looks like a specific enough classification for this subject.
    • bbor 3y ago
      I think this proves the point of who you're responding to: I'd say that the concern is about agential models that employ LLM elements, not LLM models on their own.
      • wsgeorge 3y ago
        > the concern is about agential models that employ LLM elements, not LLM models on their own. Rather? I'd imagine concerns about autonomous agents running amok is further into the future than the digital content manipulation potential of gen AI possible today. Heck, porn deepfakes of celebs were a problem in 2020, and right now Adobe has made gen AI generally available in Photoshop (AFAIK)
  • greenie_beans 3y ago
    same with the word "woke"
• calt 3y ago
  Obligatory relevant xkcd from 2018 https://xkcd.com/1968/ https://xkcd.com/1968/
• rimeice 3y ago
  Very much agree with this. If the signatories believed this, they would shut down development. We can be conveniently distracted from large societal disruption such as huge changes in the job market from automation if there’s a media frenzy on the less likely and still hypothetical extinction AI.
  • kalkin 3y ago
    What do you think of the calls for regulation or licensing of AI?
    • JumpCrisscross 3y ago
      > What do you think of the calls for regulation or licensing of AI Misdirection. We see a generic call for regulation, or unrealistic call for a global pause with no answers to how it would be coördinated or enforced. When actual regulations are put forward, they're rejected without a counter-offered solution [1]. [1] https://www.reuters.com/technology/openai-may-leave-eu-if-regulations-bite-ceo-2023-05-24/ https://www.reuters.com/technology/openai-may-leave-eu-if-re...
      • kalkin 3y ago
        Obviously companies are going to try to shape any regulation to their benefit. What I'm wondering is if people who see x-risk worries as distracting from real risks are in agreement with the narrative I've seen a lot on HN recently that calls for regulation are just anti-competitive attempts at regulatory capture.
        • JumpCrisscross 3y ago
          > companies are going to try to shape any regulation to their benefit There is a difference between trying to shape regulation to your advantage and engaging in bad faith. There is no indication there is real regulation that addresses the problem these folks are brining up that they would support.
• 1970-01-01 3y ago
  Don't you know this applies to all software? We tend to forget "don't be evil." https://en.wikipedia.org/wiki/Don%27t_be_evil https://en.wikipedia.org/wiki/Don%27t_be_evil
• chubot 3y ago
  Yes, and specifically Yudkowsky, Bostrom, and MacAskill are “providing cover” for AI owners by prattling on about AI “waking up” and extinguishing humanity They are grabbing headlines, and moving the conversation from the real issues, which are how AI is used in education, health care, law enforcement, securities and housing markets, the government, the military, and more https://news.ycombinator.com/item?id=36100525 https://news.ycombinator.com/item?id=36100525
  • kup0 3y ago
    I agree. I have plenty of concerns about AI, but those concerns involve real-world harms that can happen in the immediate/short-term, not just some hypothetical sci-fi mass casualty event and having AI criticisms focus on the wrong conversation really just compounds the short-term harm by shifting the conversation away Long before AI causes mass harm without human involvement, humans will find hundreds of ways to make it cause harm, and harm at scale. I do think the technology itself is part of the risk though, because of the flaws, the scale, etc inherent to its current iterations. However, maybe those are still the fault of humans for not giving it the proper limits, warnings, etc. to mitigate those things That said, it can be used for good in the right hands (accessibility tools, etc), potentially, though I'm certainly more of a doomer at this point in time.
  • echelon 3y ago
    This is the real danger. What's worse, we've already seen this story play out once before. Why aren't we talking more about it?
  • emtel 3y ago
    Please don't neglect to mention that a large number of very accomplished AI researchers share their concerns, or at the very least do not believe they should be dismissed out of hand.
    • deafpolygon 3y ago
      I'm reading that a good number of those accomplished researchers that work at these companies, that stand to gain if they ensure that their company remains at the helm of AI research. Better that than introduce competition and unpredictable results, possibly leaving them behind in a highly competitive industry with no moat.
• SpicyLemonZest 3y ago
  The author says: > Do the signatories believe that existing AI systems or their immediate successors might wipe us all out? If they do, then the industry leaders signing this statement should immediately shut down their data centres and hand everything over to national governments. The researchers should stop trying to make existing AI systems safe, and instead call for their elimination. But a different set of people called for a halt to AI research two months ago, and his response then (https://aisnakeoil.substack.com/p/a-misleading-open-letter-about-sci https://aisnakeoil.substack.com/p/a-misleading-open-letter-a...) was that these concerns are "fever dreams" and we should disregard them in favor of "serious policy debates". It seems like there's a self-reinforcing system here, where any evidence or argument in favor of existential risks from AI can be dismissed out of hand.
  • danaris 3y ago
    The author is not saying "AI systems should be shut down, because they might wipe us out". He is saying "if those people genuinely believed what they were saying, they should logically shut down their own AI systems." His position is consistent and clear: the supposed threat from AI is based in sci-fi and motivated reasoning. Indeed, even those who are motivated to spread the idea of that threat do not appear to genuinely believe it—either that, or they believe that their own right to continue to do whatever they can to maximize their profits outweighs it.
• harryvederci 3y ago
  "Guns don't kill people.. Uh-Uh. I kill people. With guns." - Jon Lajoie
• NumberWangMan 3y ago
  > Indeed, focusing on this particular threat might exacerbate the more likely risks. The history of technology to date suggests that the greatest risks come not from technology itself, but from the people who control the technology using it to accumulate power and wealth. This is where you lose me. AGI (not LLMs, but intelligence smarter than us) is qualitatively different from anything else we've ever made. The one example we have of this in history is the development of human intelligence. It did not work out well for other species, and now we own the Earth. This isn't a perfect analogy! There are some reasons why AGI may be better (we'll be actively trying to align it to human interests) but also why it may be worse (there will be pressure to make it more powerful and take off guardrails). There may be good reasons for why we'll be able to control AGIs, but "history of technology suggests..." is not one of them. And I'm not arguing that all the "human misuse" problems are not important! They're hugely important! A likely outcome, the way we're going, is that a bunch of people get really rich and everyone else becomes super poor and possibly starves, and wars break out. I just think that, after that, AGIs also end up in charge of the planet and from there we either die or we have no say in how things go from that point on. I don't object to working on the former problem, except to the degree that the solution makes the latter problem worse (like, "open source all the models", which is likely to happen at some point anyway).
  • deleted 3y ago
    [deleted]
  • logifail 3y ago
    > A likely outcome, the way we're going, is that a bunch of people get really rich and everyone else becomes super poor and possibly starves, and wars break out. I just think that, after that, AGIs also end up in charge of the planet and from there we either die or we have no say in how things go from that point on. Rich getting richer [x] Poor starving [x] War [x] Powerful gain more power [x] Based on how I already see the world, honestly, I'm not sure that's much of a prediction.
    • jjoonathan 3y ago
      It can get worse. Much worse.
• cudgy 3y ago
  My worry is that the attempt to regulate AI is masking a power grab by the large, politically well-connected tech firms to maintain control over the technology.
• catchnear4321 3y ago
  the risk is always people.
• wlitlwijatli 3y ago
  You can ALMOST take all of the arguments about gun control and replace "gun" with "AI". AI is a powerful thing, and can do great damage in the wrong hands. Any powerful thing can do damage in the wrong hands. The difference between AI and guns is right in the name. AI is intelligence. It could conceivably act on its own. A gun (usually) will not do anything except what its builder or owner commands it to do. So the problem with AI is half what the owner commands it to do and half what it decides to do on its own. And that does make it fundamentally different and potentially much more dangerous than other powerful things. As to which of those problem is the bigger problem, the people who control or what it decides to do on its own, I would say the owners are likely the bigger problem. If someone builds AI without safeguards in place or explicitly commands AI to do bad things, then we should expect a bad outcome. If owners act responsibly and ethically and build in safeguards, we should prepare for a bad outcome but expect a good outcome.
  • phillipcarter 3y ago
    > It could conceivably act on its own But it really can't today, not without a bad actor giving it specific goals and figuring out ways to make those yield real-world outcomes. Which is the entire point of the article.
    • kalkin 3y ago
      I didn't actually see where it made that argument - that AI intrinsically can't be agentic, or something like that - in this article. Can you point me to it?
    • Sol- 3y ago
      But people already try to do that, see AutoGPT and similar endeavors. Whether agentic AI will appear is not a question at all - humans will make it so just to see what happens or to serve their goals.
  • hanniabu 3y ago
    And just like guns, it shouldn't be banned/limited. Not to belittle the downsides, as they're valid, but the outcome is much worst when the access is only one sided. In both cases it leads to oppression.
• PeterStuer 3y ago
  OpenAI already had GPT-4 months before they released ChatGPT 3.5 . Everyone that has used both will know what a dramatic difference those two models represent. Furthermore, the internal GPT-4 was reportedly significantly more capable than what was eventually released. Who here with experience does not think these neutered late releases do not tilt the playing field? "Open"AI was founded assuming that premise. Talk about a 180.
  • waterheater 3y ago
    Perhaps history will show that "Open" is not an adjective but a command, as in opening the AI box.
• p0w3n3d 3y ago
  The reason AI is dangerous is stupid people who will believe it. It's merely a helper, not an oracle
• sleepybrett 3y ago
  This is the 'guns don't kill people' argument. Sure, the people in control of the gun bear responsibility, but maybe they shouldn't have a fucking gun in the first place.
  • mrangle 3y ago
    Its not. Guns aren't programmed to behave in a certain manner. The end user is in control. In contrast, the manufacturer of the AI determines its behavior. Not the end user.
    • JumpCrisscross 3y ago
      > Guns aren't programmed to behave in a certain manner One also can't replace nurses with guns to make healthcare more scalable.
  • flangola7 3y ago
    Precisely - the true danger is disempowerment of humans - all of them. It doesn't matter which monkey in the cage figures out how to pull the grenade pin. There's a mob of dreamy libertarian techbros on HN who are (often wilfully) ignorant of the dangers. I'll start worrying about military robots after we figure out how to prevent the Earth from being dismantled, which very well might transpire first, if and when there's significant architecture breakthroughs. AI does not need to be evil, it does not need to be conscious, it does not even need to be severely misaligned with human values, and it really doesn't matter if the goal of "eat all the humans" arises from a model hallucination or from a misanthropic troll in his basement typing it into his looped uncensored-LLaMAv3+plugins instance.
• dontupvoteme 3y ago
  Ask yourself this -- if all this happened during the cold war, would it have been considered anything special or different compared to other technologies or events? Or would it just have been our computing machine vs that godless commie one, and we'd all keep on keeping on trying to land on the moon or not lose our respective war somewhere? I submit that it is not anything special, certainly pales in comparison to the atomic bomb. It's merely that we live in that weird end tail of the age of ideology where strong overarching ideologies came to (and are coming to) an end, where we're all scattered to the various geopolitical winds and in the absence of a strong guiding state and vision (communism vs capitalism) become more extreme and perturbed. If you do want regulation, It should be something proper hashed out diplomatically and economically between Nation States and their respective blocs, no crony capitalist allowed, sorry. The rest of the world shan't be going along with something that economically cripples them.
  • SpicyLemonZest 3y ago
    I'm not really sure how to evaluate the hypothetical. When personal computers were invented during the Cold War, lots of people correctly realized that they were special and different and would radically change society in ways which had nothing to do with the political conflicts of the day.
• p0w3n3d 3y ago
  The most dangerous thing about AI is uneducated but allowed to make decisions people who will blindly believe the AI. However AI itself is merely a tool which should be used but with caution.
• teawrecks 3y ago
  I feel like this is such an antiquated take that people think it's novel again. If you told the guys on the Manhattan Project that in 80 years the next big threat to humanity would be AI, I think their first reaction would be, "then we had better be the first ones to get there". The assumption in an arms race is that it's all about who is holding the bigger bomb at the time. It's only after decades of scifi and Hollywood movies that people thought, "oh, we could accidentally create a bomb that wants to kill us all". Both are true.
• phillipcarter 3y ago
  A specific risk I am worried about today is using AI to power and make impactful decisions in high-risk infrastructure that people rely on. I do not want my power company making decisions about me based on a large language model that regularly gets things wrong. Not without significant controls in place, explainability/audibility, and the ability quickly reverse any bad decision. Replace "power company" with the numerous things we all rely on today and it freaks me out. People are jumping too quickly to deeply integrate this tech with everyday things, and while that's great for many use cases, it's not so great for others. You won't find the likes of Yudkowsky talking about this because they'd rather go on about their fan fiction than actually work to prevent real problems we may face soon. The fact that they've tricked some really smart people into going along with their shenanigans is extremely disappointing.
  • cmilton 3y ago
    >I do not want my power company making decisions about me based on a large language What data would comprise such a model?
    • giraffe_lady 3y ago
      You're getting into the nitpick weeds on a detail while ignoring the actually substantial argument the comment was making. They provided an example and finding a technical flaw in the example they chose doesn't invalidate the broader concern as applied to other domains.
      • cmilton 3y ago
        I asked a question for clarification.
        • fnimick 3y ago
          There are so many potential abuses in a situation like this that I find it baffling that you can't imagine any of them. "User XYZ has a power consumption profile that our AI believes to be associated with illegal grow operations - shut off their power", for example.
• mrangle 3y ago
  Otherwise known as a difference of no difference.
• lumenwrites 3y ago
  Ugh, another one of those not-very-insightful (or very well-informed) think pieces trying to sound interesting by being "contrarian" (even though they're repeating the same idea you can find under many HN or reddit threads). If you think it's dangerous when bad people control an AI, how dangerous do you think it will be when an AI that has no concept of good or bad controls itself? An AI whose values are more alien than that of any sociopath, completely orthogonal to human values? If you don't think that is going to happen, then make a convincing argument for your case, the way AI "doomers" provide well thought-out arguments for their case: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no... https://astralcodexten.substack.com/p/why-i-am-not-as-much-of-a-doomer https://astralcodexten.substack.com/p/why-i-am-not-as-much-o... https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a... https://www.youtube.com/@RobertMilesAI/videos https://www.youtube.com/@RobertMilesAI/videos
  • freeone3000 3y ago
    All of these arguments are predicated on some advancement or change that has yet to happen. Widespread adoption of the existing technology will cause the problems in the posted article.
    • comp_throw7 3y ago
      Sure is a good thing AI hasn't ever gotten better in unpredictable ways before.
    • lumenwrites 3y ago
      Honestly if feels like we're living in the "Don't Look Up" movie. The asteroid hasn't hit the Earth yet, so what are you worried about? The reason people talk about x-risk from AI before the world-ending AI is developed, is because after the world-ending AI is developed we won't be around to talk about it.
      • danaris 3y ago
• falcrist 3y ago
  This has always been the primary concern. Frank Herbert addressed it in Dune. Right after Paul is tested by the Reverend Mother, there's a conversation where she says the following: “Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.” Sadly, Brian Herbert seems to have turned the Butlerian Jihad (the ancient war against all thinking machines) into a struggle between mankind and AI rather than between mankind and itself.
• Zetice 3y ago
  Okay so if I'm the problem, what can I do to learn more about responsible use of AI? Are there books, classes, something I can do to proactively start working on being a "smart" AI user? Or, as I suspect, this is all very murky and nobody knows exactly what the challenge really is beyond, "Continue to not be an asshole." I am, right now, applying LLMs to domain specific business problems as part of my job. How do I do this safely/sanely, does anyone actually know?
  • Avicebron 3y ago
    Well prior to buying books specifically about ai you can start by thinking about every possible negative externalities your job currently encourages or actively engages in, down to the most minute detail, including something like power consumption displacing environmental concerns outside of your immediate vincinity. Take that list weigh it against your role in that and then multiply the effects of those by 10 and think about if you'd still be comfortable, multiple by 100 and repeat. Do this until you feel uncomfortable.
• seydor 3y ago
  i m surprised by how few people call for regulating Tesla, the one AI that already kills people
  • wccrawford 3y ago
    This is a great example of an AI that kills people when it's not used properly. And a great example of exactly how people are going to ignore any and all instructions on how to use the AI safely. Not all people, of course, but enough of them that it'll be at least statistically relevant.
  • danaris 3y ago
    I mean, some people are calling for that. But people generally look at that as "regulating cars", not "regulating AI." And the failure mode there is not at all what's being bandied about as a nightmare AGI scenario—there, it's an AI that has chosen to kill people, where with Tesla, it's just an extremely stupid AI that doesn't know any difference.
    • seydor 3y ago
      i m not sure it's treated as simply a car? A self driving AI can also be ordered to e.g. kill people
• bourneavent 3y ago
  The title seems obvious. I mean guns, and nuclear bombs and bio weapons don't kill people. People kill people. But this is something we all already know. I don't think we'll ever have a problem of accidentally trusting all our nuclear launch codes to an LLM.
  • ftxbro 3y ago
    > "The title seems obvious. I mean guns, and nuclear bombs and bio weapons don't kill people. People kill people. But this is something we all already know." sure > "I don't think we'll ever have a problem of accidentally trusting all our nuclear launch codes to an LLM." i would guess this could happen easier than you think
• boringg 3y ago
  I mean the article statement is just a statement about any tech in the history of time. It's a generic statement. A tool is a tool and the risk is how it is used and by whom.
• Veedrac 3y ago
  As the saying goes, the best way to convince yourself of AI risk is to look at the quality of the arguments against it.
  • cmilton 3y ago
    Agreed. Existential level risks, need existential level arguments.
• lumenwrites 3y ago
  > And why focus on extinction in particular? > runaway AI is a way off yet, and that it will take a significant scientific advance to get there—one that we cannot anticipate So an AI that may cause our extinction may be a result of a scientific advance "we cannot anticipate"? And you're having trouble understanding why people are concerned? All the problems you've listed (COVID, global warming, war in the Ukraine) are considered problems because they cause SOME people to die, or may cause SOME people to die in the future. Is it really that difficult to understand why the complete extinction of ALL humans and ALL life would be a more pressing concern?
  • ImHereToVote 3y ago
    Intellectuals are gonna intellectualize. As soon as a large enough number of people are holding an opinion, the intellectual pops up his head. Ew, how gauche, only stupid people are concerned with what most people are concerned about.
  • Capricorn2481 3y ago
    Global warming literally will kill everyone if it isn't stopped. The fact that you're more worried about AI than global warming is a real HN moment.
    • michaelmrose 3y ago
      There is no reason to believe this is so. A less advantageous climate will almost certainly in any reasonable projections kill/impoverish only some and in all probability a small minority of the human race unless we under stress decide to kill the rest. No reasonable scientists are projecting human extinction and by positing it as such you are erecting a trivially demolished straw man for the opposition to mock.
    • travisjungroth 3y ago
      Source? Every model I’ve seen from scientists is that climate change has a very high probability of killing a minority of people. I’m not acting like that’s a small amount. “Minority” would be tens of millions, hundreds of millions of people. I think it will be one of the greatest causes of human suffering. But it’s not an existential threat, it’s a different category.
• johnbellone 3y ago
  Guns don't kill people, people kill people.
• duxup 3y ago
  The thing is that’s what everyone wants to use it for BSing other humans. It’s not just big bad guys. I wrote a privacy policy with it…
• JumpCrisscross 3y ago
  It's tiring seeing people who made millions building AI prattling about doom and gloom and regulation. Scott Galloway called it the stop-me-before-I-kill-grandma defence. (Paraphrasing.) The cherry on top is when regulation is actually proposed, the act is dropped and obstructionism re-asserted [1]. [1] https://www.reuters.com/technology/openai-may-leave-eu-if-regulations-bite-ceo-2023-05-24/ https://www.reuters.com/technology/openai-may-leave-eu-if-re...
  • drcode 3y ago
    This argument can be made pretty much against anyone in any field of inquiry Any expert, scientist, or company executive who says "this stuff could be dangerous" can be accused of wanting more attention/grants/investment/etc
    • JumpCrisscross 3y ago
      > Any expert, scientist, or company executive who says "this stuff could be dangerous" can be accused of wanting more attention/grants/investment/etc No. Climate scientists aren't walking into Congress with a multi-million nest egg behind them, no tangible solutions in front and a playbook of rejecting all specific proposals ahead. That gives them credibility these AI researchers lack.
      • raviparikh 3y ago
        The richest person in the world, Elon Musk, is a climate entrepreneur who got there in part due to climate-driven government subsidies. Just because someone made money off it though does not mean that climate change is a fake concern.
        • JumpCrisscross 3y ago
          > richest person in the world, Elon Musk, is a climate entrepreneur who got there in part due to climate-driven government subsidies. Just because someone made money off it though does not mean that climate change is a fake concern. Nobody said this. If the only person arguing the dangers of climate change was Elon Musk, there would be room for reasonable skepticism. That's the difference between the AI debate and "any expert, scientist, or company executive who says 'this stuff could be dangerous'."
• Animats 3y ago
  "Indeed, focusing on this particular threat might exacerbate the more likely risks. The history of technology to date suggests that the greatest risks come not from technology itself, but from the people who control the technology using it to accumulate power and wealth. The AI industry leaders who have signed this statement are precisely the people best positioned to do just that. And in calling for regulations to address the risks of future rogue AI systems, they have proposed interventions that would further cement their power. We should be wary of Prometheans who want to both profit from bringing the people fire, and be trusted as the firefighters." I've been saying something like that for a while, but in less Olympian terms. Look at this first as a consumer protection problem. I've pointed out before a Frontier Airlines presentation directed to investors which specifically calls out AI chatbots as a tool for suppressing customer complaints.[1], page 45: "Today, high-touch. ... Avenue for customer negotiation. Tomorrow, self-service. "Chatbot efficiently answers questions, reduces contacts and removes negotiation." The coming thing, already here for some companies, is customer service where customers can never reach a human. We already see this with the big ad-supported companies - Google, Facebook, etc. They have the power to make major adverse decisions to a user with no consequences to the company. Few governments have the guts to stand up to that. The EU does, a little. Government doing this sort of thing is a similar problem, and governments are harder to avoid. The extinction worry is a distraction from the oppression worry. [1] https://ir.flyfrontier.com/static-files/c7e0a34d-3659-49cc-8069-e85aff126b87 https://ir.flyfrontier.com/static-files/c7e0a34d-3659-49cc-8...
  • NumberWangMan 3y ago
    I think of the extinction worry as an extention of the oppression worry. It would make sense, as AI capabilities grow, for it to replace most or all humans for positions within any company that aren't physical labor, including, eventually CEOs. After that ... what about when politicians that rely completely on AGIs for their campaigns and policy positions tend to beat politicians that don't? At some point, we'll put AGIs in power because a lot of small decisions that make sense on an individual level but not collectively. Once AGI is here, it will happen rather quickly unless we implement MORE draconian measures than it would take to stop AGI before it happens. Of course, this all hinges on how hard it is to make AGI. People have wildly varying estimates of this, it could be 3 years, or it could be 30 years or more. Your estimate may be on the higher side. But if we assume that and are wrong and it turns out to be really soon, we will be blindsided.
• api 3y ago
  This is an absolute no-brainer to me. Humans are the greatest threat to humans. Even if we did create some kind of autonomous sentient AI I'd still be more concerned about humans unless its behavior suggested otherwise. Humans are "in the game" with other humans, and for the time being our habitat is limited to habitable areas of Earth. That means we are in competition and are prone to power and control games with one another. The most logical thing for a superintelligent sentient AI to do would be make a ton of money and buy some rockets and go F right off to somewhere in the solar system with lots of free energy and resources. Why fight with humans over scraps of a little sand grain like Earth when there's a whole universe out there?
• dukeofdoom 3y ago
  The likely scenario is that we all each get a personal AI that monitors everything we do, and reports back on any dissenters. If you're an oppressive regime, you want to control public opinion and culture. Hence, one reason the public can't have free speech AI is that it could correctly identify who's oppressing them, and misinforming them.
• fithisux 3y ago
  Exactly. Idols say what the priests want.
  • rcktmrtn 3y ago
    > Idols say what the priests want. That's only partially true, and only at first. "Those that make them are like them, so are all who trust in them" (Ps 115). To promulgate idols is necessarily destroy one's own agency. And the deeper you go the worse it gets. I agree, I'm not scared at all of the machines themselves, but the demented framing of what they are. All these "AI apocalypse" people are so convinced that machines have the power to end death and suffering that can't help but construct these horrible hell-stories about the vengefulness of their gods. The thing that makes people like Yudkowsky effective is that their fear is based on a obviously sincere life of devotion.
• Sol- 3y ago
  Whether you think it's an extinction risk or just a problem in the wrong hands, both concerns ultimately seem to require strong regulation. Very unpopular around here ("you can't ban matrix multiplication"!) but hard to envision how a laissez-faire approach can hold bad actors in check if their abilities are amplified with such models. Will be interesting to see how global policies develop to deal with it. Probably something pretty bad will happen at some point and then the regulations will overshoot their goal.
• arisAlexis 3y ago
  Just the name of the substack shows great bias. Small things matter."But a powerful group of AI technologists"? Nope, that's academics, researchers and non profits. I'd say, these are the articles that are paid, not vice versa.
• lavelganzu 3y ago
  The headline is a perfect example of a tense-critical deepity: it's trivially true because of the "is" and would be trivially false if it instead used "will be". Best practice would be for headlines like this to always be explicit by adding "right now", e.g. "The greatest risk of AI right now is from the people who control it, not the tech itself." Once AI commonly acts on its own, things change.
• the_optimist 3y ago
  The greatest risk to people is other people, not technology.
• Eliah_Lakhin 3y ago
  I'm not sure if the current generation of LLMs (basically, chatbots) will be any of use comparing to the previous one. LLMs, as well as other recent outcomes of ANN technologies, are undoubtedly very impressive. Especially for the wide audience. But I feel like impressiveness is the only practical feature they have. When I talk to a support call centre chatbot, I feel like the only useful thing the robot could do for me is to guide me through the verbal user interface in a very verbose way. And I find it lesser convenient and reliable than just clicking a few buttons. ChatGPT can give me encyclopaedic knowledges through a verbal interface too, but I think that googling a Wikipedia article is more straightforward and, again, more reliable. I think the lack of reliability was the reason why people eventually gave up on the previous generation of these technologies as a form of computer interfaces. The same fate awaits the new generation too, in my opinion. And the LLMs can't offer anything new rather than just a new form of interfaces. They are very bad in true reasoning. Which is why, I think, they are too far from what I would call AGI. The author of the article is talking about new risks brought by AI. I personally don't see any new risks here, but I see a lot of risks in more traditional technologies no one is talking about. Modern Internet, smartphones and the way the modern computer interfaces designed in general are, in my opinion, already brought a lot of risks to us. We are slowly loosing control over computer systems. When I open YouTube I would like to organise and control the information that I consume. But instead Google prefers to choose what I should watch. They are slowly removing explicit control functions that I can use directly in favour to suggestions semi-automatically generated by them. And it's not just Google, it's a common trend. Facebook, Windows or Twitter are not exceptions. The control over the users attention as a way to influence, is a main value for big tech and for the people who control these corporations. And I think this is the risk for the humanity we are currently facing. Will the ANNs, LLMs anyhow advance this harmful trend? I don't know. I doubt so, but it doesn't really matter, because LLMs themselves is not a root of the threat. The root threat is that we, the users, are loosing control over computers that we use.
  • _popcornLurker 3y ago
    Whether its an LLM or AGI doesnt matter. We are actually dealing with belief. If people believe that a dumb computer parrot is in fact intelligent and capable of reasoning, they will use it as such. And that is the problem.
• nologic01 3y ago
  The risk is much more structural than any specific manifestation of tech. Technology advances inexorably, new domains are unlocked, creating ever more powerful abilities. The process is maybe even "exponential" - in the combinatorics sense. Developments are thus capable of producing an ever increasing range good or bad outcomes. But social technology, broadly defined, remains stagnant, actually regressing (if we take the deceitful and manipulative social media or the general (geo)political atmosphere as evidence). That gaping discrepancy between technical ability and societal wisdom will not sustain for much longer.
  • mrguyorama 3y ago
    Maybe we should stop pretending that computers can fix problems that have nothing to do with computers. This entire forum is a testament to people who are so up their own ass they think we can "innovate" our way out of societal issues, as if the problem with the Luddites was that they didn't know javascript.
• quadcore 3y ago
  The greatest risk of AI is by far riots from people. Once they flat-earthed it, it's gonna be a lot of fun. Watch the people who control it go underground [0] [0] Sam Altman on how to survive a nuclear war
• ypeterholmes 3y ago
  "The history of technology to date suggests that..." This logic fails because we have never had something quite like AI.
• RagnarD 3y ago
  I've been saying this for awhile now. The real risk isn't Colossus. It's governments spending billions of dollars to create enormous datacenters running the largest LLMs (and beyond) and using them to process the databases that Snowden warned about. You can be completely certainly that governments have no interest in making their own systems "safe". They'll be used - after ingesting vast amounts of data on everyone - to ask how to gain totalitarian control in the fastest yet least resistance way. How to target specific individuals in various ways, including assassination which looks like an accident or "natural causes".
  • worik 3y ago
    The risk of the state doing that, in western "democracies" is variable country to country, but lower than the the risk of private corporations doing it. Our governments, to varying degrees, are under democratic control. Private corporations are not
    • RagnarD 3y ago
      You mean the way Snowden's revelations led to illegal databases being shut down? Oh wait, they weren't?
    • krupan 3y ago
      They are all going to do it, but governments have far scarier powers when it comes to what they can do with the information they'll learn from prism-llm
      • mrguyorama 3y ago
        If an LLM can magically invent a way to kill someone with plausible deniability, that is just as dangerous for a private company or citizen to have as a government. I would argue worse actually, because most CEOs seem to believe if you can murder someone and get away with it and it would benefit your business, you are legally required to do it.
      • worik 3y ago
        > They are all going to do it, but governments have far scarier powers when it comes to what they can do with the information they'll learn from prism-llm No. Not to you and I. The government may spy on you, routinely, but it does nothing to you with that data. Private power routinely spies on you too, and has an agenda about how to use that data to their advantage, not to yours.
• arisAlexis 3y ago
  How are these kind of articles different than the covid conspiracy that was about the big pharmaceuticals? This is identical. There are clear facts and then the contrarian speculation.
• deleted 3y ago
  [deleted]
• xyzzy3000 3y ago
  Any nation state with any sense will be developing AI - we see a number of countries announcing national strategies for AI publicly, and you can bet there will be other states working on this in secret. To expect these states to comply with international 'regulation' is extremely naive. And in the meantime here are we, hackers, with with our dark Web, our peer to peer systems, our open source and our encrypted communication. We can develop AIs of our own, distributed across jurisdictions. Training costs are getting cheaper, as is computing hardware. They think they can regulate all that? Come and take it from us. The horse bolted months ago, and surely the great minds at these leading firms can see this. What's the real reason behind these calls for us to set aside our curiosity and close our minds?
  • pixl97 3y ago
    >They think they can regulate all that? Come and take it from us. Sorry, you can't buy GPUs any more. And there you go, it's over for open source AI. Our supplies will dwindle and we'll be far behind the 'licensed and regulated' data centers that are allowed these 'munitions'.
    • lightsighter 3y ago
      To be fair, it's never been easier to get access to thousands of GPUs in the cloud. It might be expensive, but that is an entirely different kind of barrier. Just a decade ago, it used to be that the only way to get access to thousands of GPUs was to get access to a supercomputer at a national lab. Now anybody with enough money can rent thousands of GPUs (with good interconnects too!) in the cloud. There's certainly a limitation on it from a money perspective, but access to the computational resources themselves is not a problem.
      • tiedieconderoga 3y ago
        "The Cloud" == "Somebody else's computers" If your government passes a law saying that GPUs can only be purchased or rented with a license, as OP was suggesting, all of that capacity disappears with the snap of a finger.
• agentultra 3y ago
  Can't agree more and has been what I've been saying from the start. There's no need to imagine risks that could happen. Reality is much more banal and sinister as it is.
• deleted 3y ago
  [deleted]
• fnordpiglet 3y ago
  My concern is who gate keeps the value AI might bring. By isolating it to megacorps they will commoditize it with the least value apportioned for the most money. I don’t buy for a second AI poses an existential risk for anyone but the shareholders of Google stock. Even if it were true, I don’t trust a megacorp to not navigate a crisis with anything but total incompetence. I’ve spent too many years in FAANG to buy into their veneer of competence. By letting it be open, we can deeply understand the risks and rewards as a species and leverage the tools to their maximum. Some will do it for evil, but the vast majority will do it for good. That’s the way it always has been. Unlike nuclear weapons or flamethrowers these aren’t things made to murder. They’re not made to do anything but speak like a Dalek on command, emit half baked code, and tell racist jokes unless prompted otherwise. They could do so much more - but we will never know how much more if only Google, Amazon, Microsoft, and Facebook are allowed to develop them behind closed doors for maximal profit.
  • samstave 3y ago
    What if there were a law in place like a FOISA for AI whereby I an request the actual code/Data that caused the AI to come to its conclusion. So if an AI generated bill for service was found that I Owe $N$ - I should be allowed to see all the code and logic that arrived at that decision.
    • fnordpiglet 3y ago
      That’s not the same as giving the model to someone and allowing them to build tools with AI powering it, or the development of alternative models (which is what they’re trying to stifle). It’s less about transparency and more about putting the tools in as many hands as possible
      • samstave 3y ago
        Its funny, this is the second time in a few days I came across your /u/ and comments. THIS WAS COMMENTED IN RED OR GREEN Anyway, yeah I think models need a way to self-register upon whom uses them.. yes Creepy AF, but also needed AF. ? disagree>??
• mcculley 3y ago
  Yes. I don't know why there is so much silly talk about alignment. If alignment were possible, we would have already aligned corporations with outcomes beneficial for humanity. AI will just make corporations more powerful and they will more effectively cause unintended damage and enslave humans. We will have amplifications of Deepwater Horizon and the tobacco industry.
• lucabs 3y ago
  [dead]
• aaroninsf 3y ago
  PSA both types of risk are real. Indeed the strongest point in favor of being critical of those concerned about existential risk over plain old grift and malfeasance and incompetence, is that the leverage provided by AI makes the latter itself an existential risk.
• wolverine876 3y ago
  If AI is such a risk to humanity, why are these tech leaders developing it? What an insane choice. I don't buy the transparent, 'technological development is inevitable' argument. Make it a secret project if you're worried about enemies eclipsing you. Somehow, the public lets them off the hook for 'stepping forward' and preaching that it should be regulated, while they continue to risk the future of humanity.
• ayakang31415 3y ago
  As a computational physics researcher, I can't see any difference between DNN model and human-deduced model except DNN model works because a corpus of data is used to find unknown parameters in the model to approximate probability mass function within a very specific system WE CAN CHOOSE and breadth of the scope WE CAN LIMIT. Risks those so called AI experts are saying already existed before and could be instigated by already existing non-DNN models, and we already have laws and measures to prevent them and we are still alive
• deleted 3y ago
  [deleted]
• hollerith 3y ago
  "I make my living from this AI newsletter (or I hope to in the future); if AI is as dangerous as some say, then it'll be banned, and I'll have to figure out some other way to make money, so I am going to say a few words about the unlikeliness of that possibility, then ignore it, but I am totally down for a long public discussion about which monkey or small group of monkeys might end up with too much control over the tech."
• fooker 3y ago
  I have heard an appropriate analogy for situation. Imagine a confident ten year old who has just learnt the rules of chess or Go. Could this person explain or quantify how the best player in the world might defeat them?
• LatteLazy 3y ago
  The greatest risk of AI right now is from people selling it, to investors/customers, since it does not exist. It's the same type of fad we have seen with crypto, self driving cars, quantum computers etc. FOMO plus panic plus zeitgeist.
• chemmail 3y ago
  So are you telling me it's not guns that shoot people, it's people that shoot people!?!?!?!
• neycoda 3y ago
  Um, if AI is allowed to learn like a person, then it can become "the people that control it".