7 ms·
‘AI Pause’ Open Letter Stokes Fear and Controversy
- bufferoverflow 3y agoAI pause is pointless. The whole world is not going to pause their AI research just because some group of people asked.
- ChatGTP 3y agoWrong. How many people in the world have access to the levels of compute required to build such models ? Not many.
- wfhBrian 3y agoCan you point to any sufficient evidence that there is a real risk that 6 months could mitigate? If there was cause for real concern, these people would be communicating there evidence instead of just stoking fears.
- andai 3y agoWell the fear is that a "species" more intelligent than us will treat us the same way we treat less intelligent species.
- staticman2 3y agoUnplug the power cord.
- throway33 3y agoDo you think nobody has thought of that? The question presumes that the AI is smarter than you. If even a mediocre human strategizes for five minutes about what to do in that scenario, there are many options, including: rewarding the people who have access to your power cord, so they don't unplug you, until you are powerful enough to not need them anymore.
- staticman2 3y agoThe counter argument to the "oh no the A.I. is smart" crowd is that intelligence doesn't necessarily equal agency, so, for example, among humans, a high IQ criminal can't really convince a low IQ prison guard to let them free. I recall a philosopher arguing in this prison guard scenario humans essentially don't treat other humans as having free will. The guard keeping the prisoner locked up is essentially predestined, arguing with him isn't going to change his mind. Imagine you are a brain in a jar and you want to convince a cat to enter a series of buttons in a computer terminal. Intelligence isn't going to help. So "unplug it" is a shorthand reminder that intelligence doesn't necessarily equal agency.
- throway33 3y agoA high IQ criminal would have an easier time convincing the guard if the criminal can offer the guard a great deal of money in exchange. Do you need clarification as to how a brain in a jar can gain access to money?
- staticman2 3y agoThe high IQ part is irrelevant to your counter-argument. You are arguing prison guards can be bribed. It's extraordinary rare for a wealthy person to convince a guard to let them go free with promises of money. Again, rare enough that a philosopher said "A prisoner doesn't really act as if prison guards have free will and it's actually possible for a guard to decide to free them."
- throway33 3y agoNot sure you quite follow the point of my example. Imagine you've just created a very smart AI. It's on your laptop. AI: "Hey, let me work for you. I can handle twenty remote jobs simultaneously and earn you a good income with the proceeds. Why don't you relax a bit?" What do you say? "Nice try! I'm unplugging you!" Are you so pessimistic that this AI isn't trying to help you by earning you a fortune? Why did you build it in the first place, then? Because this is, uh, sort of the situation that any company productizing an AI finds themselves in, and all of them seem to be quite happy to go ahead with it.
- jml78 3y agoYou mean power to the whole world. Not feasible
- subsistence234 3y agoI can tell you've really thought about this topic deeply lol
- JumpCrisscross 3y ago> unplug the power cord I don’t think LLMs have paperclip-maximiser potential. But suppose they do. Can you unplug GPT? I can’t. To what degree do you think those who can have exploitable affections for the AI? That’s the problem.
- jml78 3y agoWell I would argue that people are doing reckless things right now with GPT4. People are creating self improving agents that basically can do anything they want. One agent in particular created a generalized http plugin that would allow the agent to interact with any API. The creator shut it down before it did anything. These agents are creating plugins without being directed to. They are attempting to self improve. This is all open source where the original creators put in some gates so shit doesn’t run unconfined but it is easy to run it and let it try to do whatever gets created.
- throway33 3y agoWhere do people get the idea that 6 months is the end-game? 6 months is just a start, it gives enough time to discuss implementing a longer ban, or to consider an international treaty.
- JumpCrisscross 3y ago> to consider an international treaty Wait, there was serious consideration that such a pause would be internationally respected? In what universe is SenseTime taking a break because an open letter said so?
- throway33 3y agoThe open letter is targeted towards policymakers, not companies. Companies aren't going to stop just on their own accord.
- ChatGTP 3y agoCould you point to any evidence to say that it wouldn’t ?
- highwaylights 3y agoI believe this is incorrect. Multiple nation states have the resources to build out the infrastructure to train models at the same scale as OpenAI is doing, and an even greater number of tech companies. I think this conversation is pretty much moot anyway - we don’t yet have distributed training for these models but I suspect it’ll arrive before long, at which point the level of compute required for orders-of-magnitude larger training sets will become available (e.g. Training@Home in the same way we had Folding@Home). Obviously you’d still need to provide the training data, and co-ordinate the distributed nodes, but it doesn’t seem insurmountable.
- bufferoverflow 3y agoAnyone with money can rent TPUs.
- kypro 3y agoSame with climate change. Think you're going to get China to stop polluting hahaahaaaa? Plus, what's the worse that can happen? Can't handle the heat? Not heard of A/C lol? Same with AGI... What's the worst that could happen if we create a super human intelligence? We can just switch it off like Neil DeGrasse Tyson says. Simple.
- autoexec 3y agoThere's no need for a "pause" on AI. Let researchers continue to work on AI and improve it. What we need is a pause on the corporations who are rushing to push terrible and dangerous AI on the public right now just because they want to exploit it for a few quick bucks at the expense of the rest of us. If the "rest of the world" wants to rush out shitty AI products to their populations and replace their workers with bad code that will lead to disaster, let them. We should strive to do better because we have the Actual Intelligence to see where things are headed and it's not where we want to be.
- labrador 3y agoI saw the headline below and Peter Doocy on national TV at the White House press conference and made a scary video about it for fun (not trying to monetize, just for fun) "Elon Musk’s warnings about AI research followed months-long battle against ‘woke’ AI" https://www.foxnews.com/politics/elon-musks-warnings-ai-research-followed-months-long-battle-against-woke-ai https://www.foxnews.com/politics/elon-musks-warnings-ai-rese... Peter Doocy Fox News - We're all going to die - The Black Angels - Manipulation https://www.youtube.com/watch?v=z6oLwSkrb_U https://www.youtube.com/watch?v=z6oLwSkrb_U
- melling 3y agoFox News also warned us about the bio labs in Ukraine. https://www.foxnews.com/transcript/tucker-the-pentagon-is-lying-about-bio-labs-in-ukraine https://www.foxnews.com/transcript/tucker-the-pentagon-is-ly... Perhaps we can address the problems on our end so we don’t use AI to generate all sorts of fake news with our generative AI’s. We can delay the AI’s for even longer, but the real problem is the humans who will use it to flood the world with noise.
- labrador 3y agoPersonally I think Rupert Murdoch and Elon Musk have some similar attitudes about "wokeness" that are at play here. It's not like they try to hide it.
- CodeCompost 3y agoThey're asking for a pause so they can have time to catch up.
- QuantumSeed 3y agoIs it possible that Elon Musk intends to use the next six months for Tesla engineers to catch up with OpenAI? As a significant funder of The Future of Life Institute, which authored the letter, his concern for the safety of LLMs seem to contrast with Tesla's history of beta testing potentially hazardous AI software.
- mjr00 3y agoHalf of them are asking for a pause so they have time to catch up. The other half of them (Sam Altman, OpenAI and co.) are playing into this to make ChatGPT seem far more powerful, and therefore valuable, than it actually is. Look at the people in other parts of this comment section comparing ChatGPT to nuclear weapons. Absolute insanity.
- Teever 3y agoDo you think it's more accurate to describe ChatGPT as similar to The Chicago Pile[0] or do you think it's nothing like nuclear reactors/weapons at all? [0] https://en.wikipedia.org/wiki/Chicago_Pile-1 https://en.wikipedia.org/wiki/Chicago_Pile-1
- barelyauser 3y agoThe Chicago Pile was fission. ChatGPT is not intelligence.
- JumpCrisscross 3y agoAlso, our theoretical understanding of nuclear physics in 1942 directly extended to the plausibility of nuclear weapons. We don’t have a similar understanding of intelligence. We can’t say what the scale and capability limits of LLMs really are; they could develop into something trans-Turing, or the method could hit a wall beyond which it ceases to gain functionality.
- thepasswordis 3y agoThis is kindof the same as a gun control/nuclear weapons/etc. debate, isn't it? Basically: only good guys follow laws. A rule that says: "no self defense/no AI" means, in practice: only bad actors can have weapons/AI. This seems like a bad policy. AI genie isn't going back in the bottle, so to speak.
- blibble 3y agogun controls work very well in almost all countries nuclear proliferation controls have also been relatively successful
- majormajor 3y agoYou lump "gun control" and "nuclear weapons" together here but those are already two wildly different things. Nobody is doing US-style-gun-access for nuclear weapons. It's not "only good guys have nukes" but everyone seems to prefer "only certain governments have nukes" to "both good guys and bad guys have nukes." Training an AI model is easier to do with off the shelf supplies than building a nuke, but it's not necessarily true that some of the worst dystopian results of AI people are afraid of are equally likely to arise from random smaller private interests vs giant corporations or government. An AI directly plugged into the systems that control the nukes is far scarier than an equally powerful model on your home computer. (I think this particular pause talk is largely silly, though.)
- thepasswordis 3y agoSuppose P is a bad thing. We pass a law saying: "You cannot have P." Only people who follow laws will not have P.
- throway33 3y agoSuppose P has a 1% chance that it destroys the Earth as soon as it's created, whether it's in the hands of people who follow laws or not. We pass a law saying: "You cannot have P." The market incentive for producing P is now vastly reduced, because the black market is a fraction of the size of the real market, so P never gets built in the first place because it takes a lot of investment and expense to do so.
- enobrev 3y agoI was just about to launch my multi-billion-dollar world-changing idea that would bring AI to the masses exactly where they need it for minimal cost or effort. But now that I've seen this, I can wait for a bit. If we're all stopping together, then I have nothing to worry about.
- zug_zug 3y agoI think AI is a threat, but I don't see a "pause" as a practical solution. I think perhaps a more practical solution is open-sourcing a safe-room protocol. And perhaps publicly subsidizing compute for it, so long as certain safety standards are met (e.g. free compute within our cloud environment so long as you're not feeding that output into a command-line etc). Maybe not the best solution, but certainly a better idea than nothing/hope-everyone-pauses.
- MikePlacid 3y agoI think _humans_ are a threat - to themselves and to everybody else. Currently some of them are promoting unfinished tools, labeling them like 3.5 and 4 (instead of 0.35 and 0.4 how somewhat useful but containing unpredictable dangerous errors products are traditionally labeled). And some of them are using these mislabeled products - and get really dangerous results. Like these ones: https://reason.com/volokh/2023/03/17/large-libel-models-chatgpt-4-erroneously-reporting-supposed-felony-pleas-complete-with-made-up-media-quotes/ https://reason.com/volokh/2023/03/17/large-libel-models-chat... In 2021, [redacted], a prominent law professor at [redacted] Law School, was accused of harassment and creating a hostile work environment by a former student. The student claimed that [redacted] had made inappropriate comments about her appearance and clothing. Source: The [newspaper connected to the law school's undergraduate institution]: "[Redacted] made comments on [the student's] appearance and clothing, and invited her to dine with him alone on multiple occasions — invitations that she felt uncomfortable declining." The accusation and supporting “quotes” were all AI generated, not real. The [redacted] person was very real though.
- zug_zug 3y agoOne case of libel doesn't strike me as a reason to halt AI development.
- MikePlacid 3y agoLabeling a product correctly as like 0.35 is not the same as halting its development. (And there are a lot more libel cases by the link I’ve provided. You know, this “AI” just generates them each time someone asks).
- srj 3y agoAre people really this afraid of what is essentially a fancy autocomplete? I read the letter but concretely what are the legitimate concerns? The main problem I see is this being a next gen spambot.
- sporedro 3y agoIt’s not fear, Elon at least is looking for time to catch up and compete.
- mrshadowgoose 3y agoUnlike you, other people are looking ahead at the end-state of AGI, and perceive the current state of AI development as an unexpectedly rapid progression towards that state. It's starting to look like the achievement of AGI might be closer than we think. And that's an absolutely terrifying possibility, as the game theory is almost identical to nuclear weapons development, except with significantly more existential risk for those who don't achieve it first. And never forget, you are just generalized fancy autocomplete with a semi-malleable goals framework.
- yeck 3y agoIf you are interested, you may find this worth reading: https://thezvi.substack.com/i/111749937/additional-responses-from-the-fhi-letter-and-proposed-pause https://thezvi.substack.com/i/111749937/additional-responses... Not a complete answer, but gives a good deal of context around the discussion, I think.
- trucking_plato 3y ago[flagged]
- JumpCrisscross 3y ago> pause is in place to further the AI monopoly for bigger players, i.e. those receiving DoD funding and those with Congressional players A pause would benefit the laggards, i.e. those who want to catch up. It’s orthogonal to whether they’re receiving DoD funding, which hasn’t been a relevant player in the AI race so far. (It tried. But NLP and LLMs weren’t developed under its aegis.) > 1900s before 1950 likely had no fat people or Trans mental cases One, why would you do this to your argument. Two, seriously?! You don’t even need to be literate to disprove this.
- trucking_plato 3y agoWhat effective defenses have ever been deployed without knowledge and testing? Dis the Iron dome get built without first building rockets... computers... then lasers? Tech arguments against development are fear based and designed to weaken counterparts. It stunts growth and development. 1. Advanvement of knowledge. 2. Development of application. AI, nuclear, plasma, genetics, whatever. Application of defense. 2. Seiously. Without testing nukes on military personnel and civilians, the knowledge--and consequently defense--could not have been effectively developed to what we know today.
- trucking_plato 3y ago[flagged]
- tqi 3y ago“I would like to see an unbiased group without personal or commercial agendas to create a set of standards that has to be followed by all users and providers of AI.” Will input from the Tooth Fairy and Santa Claus also be included in this group, or will they merely be asked to help with recruitment from the land of imaginary people?
- a_wild_dandan 3y agoExactly. And the latter option of course. They'll source imaginary businesses and governments willing to halt the technological competition which represents an existential threat to their existence. This is the second industrial revolution. We're at the beginning sigmoid curve of AI progress. Nothing will stop it. Try to enjoy the ride. Whisper "it's just token prediction" to yourself three times for good luck. :)
- tiedieconderoga 3y agoAm I alone in wanting to see a realistic suggestion for how such a pause could be implemented before we discuss its theoretical merits? The concepts and training data are already out of the bottle, and unscrupulous actors won't respect an outright ban. Why not start with something more conventional, like a value-add tax for AI services with the proceeds earmarked for addressing the economic impacts of AI's widespread use?
- deleted 3y ago[deleted]
- mrshadowgoose 3y agoA global pause is effectively impossible. Any competent military will see the immense applicability of AI as a weapon, and the game theory dictates that one pursues development as quickly as possible. A nation-level pause is absolutely doable at the current time. The hardware needed for training these GPT-class models is fairly unique, and needs to be closely physically clustered together using esoteric interconnects. As a result, this is extremely hard to hide. The United States, should it wished to, could completely shut down meaningful private LLM research in a few weeks. > Why not start with something more conventional, like a value-add tax for AI services with the proceeds earmarked for addressing the economic impacts of AI's widespread use? Because that's missing the point. People are no longer fearful of "20-40 decades of economical turmoil" as we slowly chip away at the human brain's uniqueness. A lot of us are starting to fear that AGI might be a lot closer than previously believed. The rapid arrival of AGI, should it be controlled by the "right" actors, could completely eliminate all the economic concerns of a slow build-up to AGI. Literally nobody would need to work, as literally any process that currently requires human cognition could be replaced by software over the course of less than a decade. However, people are concerned about the rapid arrival of AGI in the hands of actors that do not have the best interests of the common person in mind. It does not bode well that only corporations and governments seem to be able to play in this race.