6 ms·
Yoshua Bengio: We need a humanity defense organization
- ChatGTP 3y agoSeriously reading this article and thinking about where we headed really puts every moment into perspective and makes me realise it’s precious. Every time I look at children, and the wars going on is a reminder it could all just be wiped away so quickly. Enjoy while it lasts… Maybe this revolution will end well but man it seems like it’s going to be a wild ride.
- ThinkBeat 3y agoOpenAI and the rest of the companies doing a lot of their activity and business out on the open should not be the primary worry when it comes to AI. Rather think about what NSA,CIA,Pentagon and similar organizations are doing in private that we will never hear about. (unless Snowden two comes along) Whatever ethical guidelien and assurances and limits that are imposed on consumer / business related AIs is godo, but it won't impact the intelligence and military systems.
- mistrial9 3y agolead individual there also calls for collective response in "International Institutions for Advanced AI" [0] [0] arxiv:2307.04699v2 July 2023
- deleted 3y ago[deleted]
- wccrawford 3y agoDo people not realize that "godfather" implies that the person wasn't directly involved with bringing that thing into the world? That would be just "father". That, or they're the head of the AI Mafia.
- jedberg 3y agoA godfather is classically a close family friend who helps you get ahead in life by treating you like one of their own children. In this sense the godfather of AI would be someone who is influential in the success and adoption of AI without being directly involved in the creation of the technology.
- lcnPylGDnU4H9OF 3y ago> AI Mafia The current state of AI marketing has me stuck on the marvelous name of MafAI.
- tomjakubowski 3y agoMafIA: Making a fantastically Intelligent Artifact
- mochomocha 3y agoIt's apt in Yoshua's case though, as he hasn't made any significant contribution himself. I'd argue his brother during his tenure at Google has had a much bigger impact.
- skummetmaelk 3y agoBut what does Schmidhuber think?
- taylorius 3y agoHe's waiting until the AI christening. When the Godfathers are all lined up, Schmidhuber will dramatically burst through the church doors, brandishing a sheaf of papers from the mid 90s.
- cs702 3y agoOf course, at the Apocalyptic End-of-Times, we'll all get what we deserve: More Schmidhubering! In all seriousness, he deserves more credit than he routinely gets.
- taylorius 3y ago"In all seriousness, he deserves more credit than he routinely gets." 100%, he's done so much amazing, groundbreaking work! In all fairness he should be included in the "Godfathers" list, for sure.
- esafak 3y agoHe held an AMA a few months ago: https://www.reddit.com/r/MachineLearning/comments/13q6k4a/interview_with_juergen_schmidhuber_renowned/ https://www.reddit.com/r/MachineLearning/comments/13q6k4a/in...
- strikelaserclaw 3y agoWhy can't they just call these people the 'Father of AI'
- Zambyte 3y agoBecause his name isn't John McCarthy.
- blackmesaind 3y agoBecause his name isn't Geoffrey Hinton
- thomastjeffery 3y agoAI does not (yet) exist. You can't father something that doesn't exist.
- lsy 3y ago"I have the impression that it might not take a lot to fix what’s missing." The history of the field is full of people saying this, and then eventually realizing that it does indeed take a lot to fix what's missing. Right now I think we have stumbled on a novel and interesting result, which is that if you statistically collate all of the text in the world then a computer can also generate text that encodes many of the statistical relations present in that text. But I have yet to hear a plausible argument for how this translates to intelligence except that people have an impression that it's "close", never mind that we don't have any stable definition for what it's close to.
- wddkcs 3y agoIt's not close to, it is the instantiation of a collective intelligence. That the corpus was produced atemporally, doesn't matter, the responses are aionic. Any response is a slice of the past, present, and future, in the same way our conscious reflections are. Is the probabilistic foundations of our conscious process disqualifing for intelligence?
- lcnPylGDnU4H9OF 3y ago> But I have yet to hear a plausible argument for how this translates to intelligence I don't really mean to argue one way or the other but I also haven't heard a plausible argument in the other direction. Usually it's just a description of what's happening in the model as if to prove that it's definitely not intelligence but I haven't seen or heard an explanation of what intelligence is and why the actual processes of LLMs are definitely not that.
- deleted 3y ago[deleted]
- amalcon 3y agoIt calls to mind a memorable exchange in the Star Trek: The Next Generation episode "The Measure of a Man". In that episode, Picard asks another character to prove that he, Picard, is sentient[0]. Likewise, I might ask how to prove that humans are intelligent (subtext: in ways that LLMs are not). The most obvious delta here, to me, is generalization. Humans appear to be able to reason over questions that are less similar to what they have seen before, compared to current LLMs. E.g. making ChatGPT play chess was a bit of a joke meme for a while, since it routinely makes illegal moves. Your average human can be quickly taught to play chess well enough to mostly avoid that. ChatGPT can recite the rules of chess very accurately, but it seems unable to apply them to actual games. We've seen other projects that can learn the rules and strategy of games like chess, but those projects are (despite using neural nets under the hood) structurally pretty different from LLMs. They also generally have no capacity to describe the rules in human language. Note that this is not a complete argument. There exist humans (in possession of intelligence) who cannot meaningfully learn to play chess either. E.g. my friend's 1-year-old child is obviously intelligent in a very literal sense, but does not have the patience to learn chess. I can't evaluate whether she is intelligent enough for that, because there is this other constraint preventing it. This is only a framework to think about this. If you think LLMs are capable of intelligence -- you should not be convinced by it in isolation. [0]- It makes sense in context. He's trying to establish a satisfactory test for sentience.
- clubm8 3y agoWe did this before, it was called "Lifeboat", they took a bunch of money from Jeffrey Esptein and people started pretending they hadn't been affiliated or whatever when they started throwing sex traffickers into cells. I am tired of people creating precarity then showing up to solve it after years of running roughshod over this planet. "AI" is often papering over bad datasets -- you don't have the data to do so much as a student's t test, but you cater to people who only listen when they hear a "yes" after many many times being told how stuff works. [0] "The Lifeboat Foundation is a nonprofit nongovernmental organization dedicated to encouraging scientific advancements while helping humanity survive existential risks and possible misuse of increasingly powerful technologies, including genetic engineering, nanotechnology, and robotics/AI, as we move towards the Singularity. Lifeboat Foundation is pursuing a variety of options, including helping to accelerate the development of technologies to defend humanity, including new methods to combat viruses (such as RNA interference and new vaccine methods), effective nanotechnological defensive strategies, and even self-sustaining space colonies in case the other defensive strategies fail. We believe that, in some situations, it might be feasible to relinquish technological capacity in the public interest (for example, we are against the U.S. government posting the recipe for the 1918 flu virus on the internet). We have some of the best minds on the planet working on programs to enable our survival. We invite you to join our cause! The Lifeboat Foundation now has the world’s first bitcoin endowment fund" https://web.archive.org/web/20140228233556/http://lifeboat.com/ex/about https://web.archive.org/web/20140228233556/http://lifeboat.c... [1] "Jeffrey Epstein is a financier and science philanthropist. He is the Chairman and CEO of the Financial Trust Company and the founder of the Jeffrey Epstein VI Foundation. Jeffrey is a former member of the Trilateral Commission, the Council on Foreign Relations, the New York Academy of Science, and a former Rockefeller University board member. He is actively involved in the Santa Fe Institute, the Theoretical Biology Initiative at the Institute for Advanced Study, the Quantum Gravity Program at the University of Pennsylvania, and once sat on the Mind, Brain, and Behavior Advisory Committee at Harvard University. He is also a member of the Edge Foundation. Jeffrey established the Jeffrey Epstein VI Foundation in 2000 to support cutting edge science and medical research around the world. In 2003, the foundation established the Program for Evolutionary Dynamics at Harvard University with a $35 million gift. The Program is directed by Martin Nowak, Professor of Biology and Mathematics at Harvard. The Program studies the evolution of micro-biology through the lense of mathematics. Since its establishment, the Program has made pivotal advances in the treatment of cancer, HIV, and numerous infectious diseases. The Jeffrey Epstein VI Foundation is based out of St. Thomas in the US Virgin Islands and plays an increasingly active role in supporting youth and early education programs across the US Virgin Islands and the United States." https://web.archive.org/web/20140228233556/https://lifeboat.com/ex/bios.jeffrey.epstein https://web.archive.org/web/20140228233556/https://lifeboat....
- renewiltord 3y agoThe OpenAI safety crew seems to create stuff. Everyone else in the industry, Yud to Gebru are just wordcels: producing voluminous amounts of text with inside jargon. A side show.
- hollerith 3y agoWell, yeah, of course if you think creating AIs is bad for the planet, then you're going to choose not to create AIs and not to help others do so.
- deleted 3y ago[deleted]
- renewiltord 3y agoI'm not saying they have to release AI. I'm saying that they're not producing anything except blogposts. Even slowdown policy would be something since it would stall AI development. Instead, they're the same as all the environmentalist non-profits that build awareness.
- hollerith 3y agoI replied to the comment I wanted to reply to, not what you actually wrote. Specifically, where you wrote, "the OpenAI safety crew," I chose to see "the OpenAI crew". Sorry about that. (Still, it'd be better for the planet IMHO for OpenAI to get shut down, safety crew and all.)
- renewiltord 3y agoAll good. Easy misread to make.
- AndrewKemendo 3y agoI had lunch with Yoshua at the 2013 AGI Conference in Laval From that experience and a few others demonstrated to me that he - along with Sutton - are who I think actually have the background, empathy, capacity and lack of financial masters to actually lead AGI discussions.
- avgcorrection 3y ago> are who I think actually have the background, empathy, capacity and lack of financial masters to actually lead AGI discussions. What does double actually mean?
- nescioquid 3y agoTwo adverbs ('actually' and 'actually') are modifying two different verbs ('have' and 'lead').
- avgcorrection 3y agoThanks, actually.
- darepublic 3y ago> D’Agostino: But how could a machine change the climate? ^ Bengio's reply to this, loosely summarized is: "the internet / grid could go down" and "ai could manipulate people" I am more worried about hunter drones | drones that self manufacture. I am less worried about ai manipulation, and the telecommunications / electronic infra going down seems like something that doesn't require AI and is only blocked now by other factors.
- kleiba 3y ago> “Previously thought to be *decades* or even centuries away, we now believe it could be within a few years or *decades*,” Bengio told the senators. Good one. :-)
- cs702 3y agoHe's probably right. But part of me wonders: Shouldn't we first prioritize other things like, you know, reducing hunger caused by lack of food, reducing ignorance due to lack of access to a good education, increasing human rights in areas that lack them, promoting stable political institutions in places that lack them, engineering solutions to prevent or delay catastrophic climate change, and so on? There are so many unsolved problems that are urgent right now.
- ilaksh 3y agoYes, those unsolved problems (or similar to them) should have been or were priorities for most of human history. Humanity has shown that, so far, it is incapable of or unwilling to solve them. Existential AI risk has to be added to the pile and also taken seriously. Short term existential AI risk is probably from a race towards superintelligent (hyperspeed) military AI weapons command systems. It is a lever that enhances the existing types of risks. The potential for military conflicts are accelerated by the types of problems you mention. All of these challenges are interrelated. Integrating effective and accurate information collection, dispersal and application into government and society in general may be key to progress. Maybe deployment of AI is a crux that could be very helpful or make things much worse. Without an integrated global political system, AI could make military conflicts even more existentially threatening. The US and China may continue to race ahead, deploying ever more powerful AI, controlling larger and larger swarms of military assets with more and more autonomy.
- worik 3y agoI am frustrated We do face an existential threat, but this is not it We face an existential threat from capitalists and greed heads that want to burn the world for profit. This is a real looming catastrophe that is entirely avoidable Except for greed The threat from AI is entirely hypothetical
- pjerem 3y agoWell, I’m not sure those treats are unrelated. I had an interesting conversation with … ChatGPT. You know what ? ChatGPT is prompted to convince you that it is inoffensive thanks to OpenAI’s safeguards, precautions, and research on ethical AI. But ask him what would happen if a malicious organization took control of OpenAI. Ask him if it would represent a threat for humanity and, interestingly, it will answer that yes, that would be a big threat for humanity. GPT is convinced / prompted that OpenAI are the good guys. Interestingly, I don’t share this view.
- PeterisP 3y agoI think the key difference is whether people mean it literally or metaphorically if they say that something is an "existential threat". Inequality and oppression and war and climate change and surveillance states and slavery and all kinds of other things may result in a dystopian future, but all those are obviously not literally existential threats, as humanity would keep existing, only in fewer numbers and much worse conditions. On the other hand, there all kinds of events that are much less likely than those former ones, but are literally existential threats for humanity - like major asteroid strikes or the worst case scenarios of stronger-than-human AI, or perhaps some pandemics or nuclear war might cross that threshold iff they can wipe out all of humanity, not "just" most of it.
- deleted 3y ago[deleted]
- slibhb 3y agoThis dude spent his career trying to create AI and then, after not creating AI, has now decided that AI is bad and that we need some kind of far-fetched global treaty to prevent other people from creating AI. It's incredibly silly. Even if the risk is real, a "humanity defense organization" is not going to happen. Even with nuclear weapons, it was necessary to use them before we agreed not to. If AI is a danger it's going to be the same situation.
- moffkalast 3y agoIf you take into account that they subscribe to the "nukes will ignite the atmosphere and kill everyone on first detonation" in the sense of your example then it makes a bit more sense I guess. But like with nukes I really doubt the first AGI will go full Skynet in two seconds. Everyone knows you don't go full Skynet.