8 ms·
International Scientific Report on the Safety of Advanced AI [pdf]
- joebob42 2y agoHow would they know? We don't have general AI but it's written as if they already know what it will be, how safe it will be, etc. I think it's an important topic to discuss and consider, but this seems to be speaking with more knowledge and authority than seems reasonable to me.
- heyitsguay 2y agoFrom looking at the summary, I think it's a bit more measured than this statement implies. They talk about concrete risks of spam, scams, and deepfakes. They then go into possible future harms but couched in language of "experts are uncertain if this is possible or likely" etc.
- mikpanko 2y agoI believe by “general-purpose AI” the report doesn’t mean AGI.
- ben_w 2y agoWhich is one of those cases where I briefly want to reject linguistic descriptivism because to me the "G" in "AGI" is precisely "general". But then I laugh at myself, because words shift and you have to roll with the changes. But do be aware that this shift of meanings is not universally acknowledged let alone accepted — there's at least half a dozen different meanings to the term "AGI", except one of them requires "consciousness" and there's loads of different meanings of that too.
- api 2y agoYou’ve hit on a giant thing that bothers me about this discourse: endless rationalistic discourse about systems and phenomena that we have no experience with at all, not even analogous experience. This is not like the atomic bomb. We had tons of experience with big bombs. We just knew atom bombs if they worked could make orders of magnitude larger booms. The implications if real big bombs could be reasoned about with some basis in reality. It wasn’t reasoning about wholly unknown types of things that no human being has ever encountered or interacted with. This is like a panel on protocols for extraterrestrial contact. It’d be fine to do that kind of exercise academically but these people are talking about passing actual laws and regulations on the basis of reasoning in a vacuum. We are going to end up with laws and regulations that will be simultaneously too restrictive to human endeavor and ineffective at preventing negative outcomes if this stuff ever manifests for real.
- FeepingCreature 2y agoI mean, we have an entire biosphere, us included, for samples of entities of varying intelligence.
- api 2y agoThere are so many differences here vs organisms shaped by evolution and involved in a food web with each other. This is much closer to space aliens or beings from another dimension. If there are huge risks here they are probably not the ones we are worried about. Personally one of my biggest worries with both sentient AI and aliens is how humans might react and what we might do to each other or ourselves out of fear and paranoia.
- ben_w 2y ago> This is not like the atomic bomb. We had tons of experience with big bombs. We just knew atom bombs if they worked could make orders of magnitude larger booms. The implications if real big bombs could be reasoned about with some basis in reality. Well, we thought we did. We really didn't fully appreciate the impact of the fallout until we saw it; and Castle Bravo was much bigger than expected because we didn't know what we were doing; and the demon core; and the cold war arms race… But yeah, my mental framing for this is a rerun of the first stage of the industrial revolution, and it took quite a lot of harm for what is now basic workplace health and safety such as "don't use children to remove things from heavy machinery while it's running", and we're likely to have something that's equally dumb happen even in the relatively good possible futures that don't have paperclip maximisers or malicious humans using AI for evil.
- boesboes 2y agogeneral purpose AI != AGI They just means 'not trained for exactly one tasks', i.e LLMs and such and not AlhpaFold.
- joebob42 2y agoThis makes more sense, thank you. I hadn't picked up on the distinction, but I agree that's more reasonable. I still think we don't really know; it's developing technology and it's changing so fast that it seems like it's probably too early for experts on practical applications to exist and claim they know the impact it will have.
- tkwa 2y agoIt seems fine to me. When there is evidence for a certain type of current or future harm they present it, and when there is not they express uncertainty. Can AI enable phishing? "Research has found that between January to February 2023, there was a 135% increase in ‘novel social engineering attacks’ in a sample of email accounts (343*), which is thought to correspond to the widespread adoption of ChatGPT." Can AIs make bioweapons? "General-purpose AI systems for biological uses do not present a clear current threat, and future threats are hard to assess and rule out."
- casebash 2y agoDid you read the report? It's answer for basically anything contentious was, "views differ"
- ninetyninenine 2y agoThe biggest risk of ai can be simply characterized by one concept: This very report could have been generated by AI. We can’t fully tell.
- warkdarrior 2y agoSo what if it was generated by AI? Does that invalidate its contents in any way?
- euroderf 2y agoInteresting point. Are we already at a state where an A.I. could respond to email, phone calls, and even video calls in a convincing way ?
- deleted 2y ago[deleted]
- ben_w 2y agoEmail definitely, just have to remember to fine tune so it says "sure, I'll get on that after lunch" rather than "as a language model…". Voice calls, yes: I attended a talk last summer where someone did that with an AI trained on their own voice so they didn't have to waste time on whatsapp voice messages. The interlocutors not only didn't notice, they actively didn't believe it was AI when he told them (until he showed them the details). Video… I don't think so? But that's due to latency and speed, and I'm basing my doubt on diffusion models which may be the wrong tool for the job.
- ninetyninenine 2y agoNo it does not. This is the scary part.
- semi-extrinsic 2y agoDo you mean scary as in "it is scary that apparently intelligent and sane people are wasting time and money on producing documents that consist purely of meaningless fluff"? If so, I agree.
- thegrim33 2y ago"At a time of unprecedented progress in AI development, this first publication restricts its focus to a type of AI that has advanced particularly rapidly in recent years: General-purpose AI, or AI that can perform a wide variety of tasks" Is "general-purpose AI" supposed to be something different from "AGI" or from "General Artificial Intelligence"? Or is it yet another ambiguous ill-defined term that means something different to every person? How many terms do we need? It's funny that they claim that "general purpose AI" has "advanced particularly rapidly" even though they didn't, nor can't, define what it even is. They have a glossary of terms at the end, but don't bother to have an entry to define general purpose AI, which the entire report is about. They closest thing they include for defining the term is "AI that can perform a wide variety of tasks".
- krisoft 2y ago> It's funny that they claim that "general purpose AI" has "advanced particularly rapidly" even though they didn't, nor can't, define what it even is. I'm confused about what you are missing. You are quoting their definition. "General-purpose AI, or AI that can perform a wide variety of tasks". That's their definition. You might not like it but that is a definition.
- GPerson 2y agoA corporation, a question and answer website, a Minecraft game, and basically any sufficiently complex system could all be General-purpose AIs by that definition though. Having an overly general and useless definition is not better than having no definition in my opinion, so I see where the OP is coming from.
- FeepingCreature 2y agoI have seen people call corporations AIs. This is not obviously wrong.
- jprete 2y agoIt's general-purpose AI as AI is commonly understood, which is probably specific and useful enough for such a report.
- hiAndrewQuinn 2y agoRecursively self-improving AI, of the kind Nick Bostrom outlined in detail way back in his 2014 book Superintelligence and Dr. Omohundro outlined in brief in [1], is the only kind which poses a true existential threat. I don't get out of bed for people worrying about anything less when it comes to 'AI safety'. On the topic: One potentially effective approach to stopping recursive self-improving AI from being developed is a private fine-based bounty system against those performing AI research in general. A simple example would be "if you care caught doing AI research, you are to pay the people who caught you 10 times your yearly total comp in cash." Such a program would incur minimal policing costs and could easily scale to handle international threats. See [2] for a brief description. If anyone wants to help me get into an econ PhD program or something where I could investigate this general class of bounty-driven incentives, feel free to drop me a line. I think it's really cool, but I don't know much about how PhDs work. There's actually nothing special about AI in regards to this approach, it could be applied to anything we fear might end up being a black-urn technology [3]. [1]: https://selfawaresystems.com/wp-content/uploads/2008/01/ai_drives_final.pdf https://selfawaresystems.com/wp-content/uploads/2008/01/ai_d... [2]: https://web.archive.org/web/20220709091749/https://virtual-instinct.xyz/ https://web.archive.org/web/20220709091749/https://virtual-i... [3]: https://nickbostrom.com/papers/vulnerable.pdf https://nickbostrom.com/papers/vulnerable.pdf
- StableAlkyne 2y ago> Such a program would incur minimal policing costs and could easily scale to handle international threats Who ensures every country enforces the ban? How do you ensure international cooperation against a nation that decides to ignore it, against whom sanctions have no effect? What if that nation is a nuclear power?
- ben_w 2y ago> is the only kind which poses a true existential threat You don't accept the possibility that a non-improving tool of an AI system that is fixed at the level of "just got a PhD in everything" by reading all the research papers on arxiv, might possibly be advanced enough for a Jim Jones type figure to design and create a humanity-ending plague because they believe in bringing about the end times?
- wslh 2y agoClearly AI is unstoppable as it is math. I don't get how politicians or intelectuals continue to argue without understanding that simple truth. I don't know if AI will be comparable to human or organizational intelligence but know that is unstoppable. Just ranting but a potential way of defending about it is taking an approach similar to cryptography for quantum computers: think harder if we can excel on something even assuming (some) AI is there.
- hollerith 2y ago>Clearly AI is unstoppable as it is math. I don't understand this argument. It takes years to become competent at the math needed for AI. If stopping AI is important enough, society will make teaching, learning and disseminating writings about that math illegal. After that, almost no one is going to invest the years needed to master the math because it no longer helps anyone advance in their careers, and the vast majority of the effect of math on society is caused by people who learned the math because they expected it to advance their career.
- wslh 2y agoIt is very simple: powerful governments tried to stop cryptography, we know what happened. Also governments tried to prohibit alcohol, etc. it does not work. You can get them even in places such as Saudi Arabia. Are they expensive? For sure, but when it is about science that you can run in your own computers nothing can stop it. Will they put a Clipper chip?
- hollerith 2y agoThe difference between "stop AI" and "stop cryptography" is that those of us who want to stop AI want to stop AI models from becoming more powerful by stopping future mathematical discoveries in the field. In contrast, the people trying to stop cryptography were trying to stop the dissemination of math that had already been discovered and understood well enough to have been productized in the form of software. Western society made a decision in the 1970s to stop human germ-line engineering and cloning of humans, and so far those things have indeed been stopped not only in the West, but worldwide. They've been stopped because no one currently knows of an effective way to, e.g., add a new gene to a human embryo. I mean that (unlike the situation in cryptography) there is no "readily-available solution" that enables it to be done without a lengthy and expensive research effort. And the reason for that lack of availability of a "readily-available solution" is the fact that no young scientists or apprentice scientists have been working on such a solution -- because every scientist and apprentice scientist understood and understands that spending any significant time on it would be a bad career move. Those of us who want to stop AI don't care you you run LLama on your 4090 at home. We don't even care if ChatGPT, etc, remain available to everyone. We don't care because LLama and ChatGPT have been deployed long enough and in enough diverse situations that if any of them were dangerous, the harm would have occurred by now. We do want to stop people from devoting their careers to looking for new insights that would enable more powerful AI models.
- consumer451 2y agoI understand that this report is not really about AGI, but I would like to again raise my main concern: I am much more worried about the implications of the real threat of dumb humans using dumb "AI" in the near-term, than I am about the theoretical threat of AGI. Example: https://news.ycombinator.com/item?id=39944826 https://news.ycombinator.com/item?id=39944826 https://news.ycombinator.com/item?id=39918245 https://news.ycombinator.com/item?id=39918245
- tkwa 2y agoThis is like someone saying "I am much more worried about the implications of dumb humans using flintlock muskets in the near term, then I am about the theoretical threat of machine guns and nuclear weapons." Surely the potential for both misuse and mistakes goes up the more powerful the technology gets.
- consumer451 2y agoThat's fair, but to keep going with the analogy: we are currently the Native Americans in the 1500's, and the Conquistadors are coming ashore with their flintlocks (ML). Should we be more worried about them, or the future B-2 bombers, each armed with sixteen B83 nukes (AGI)? I understand that the timeline may be exponentially more compressed in our modern case, but should we ignore the immediate problem? In this analogy, the flintlocks could be actual ML-powered murder bots, or just ML-powered economic kill bots, both fully controlled by humans. The flintlocks enable the already powerful to further consolidate their power, to the great detriment of the less powerful. No super AGI is necessary, it just takes a large handful of human Conquistador sociopaths with >1,000x "productivity" gains, to erase our culture. I don't understand how we could ever get to the point of handling the future B-2 nuke problem, as a civilization, without first figuring out how to properly share the benefits of the flintlock.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- blackeyeblitzar 2y agoI feel like a lot of the arguments listed in “4.1.2 Disinformation and manipulation of public opinion” apply in a non AI world. In social media, sites like Hacker News are rare. In most places you’ll see comments that are (purposely) sharing a subset of the whole truth in order to push a certain opinion. From my perspective disinformation and manipulation of public opinion already happens “at scale”. Most social media users belong to one political side or the other, and there are lots of them, and almost all of them are either factually incorrect or lack nuance or an understanding of their opponents’ view.