14 ms·
Yann LeCun and Andrew Ng: Why the 6-Month AI Pause Is a Bad Idea [video]
- tpl 4y agoIt doesn't matter that its a bad idea. It's not implementable as policy.
- yeck 4y agoHow so? Could it not be compared to restrictions on activities like enriching uranium? Or research on biohazardous materials?
- davesque 4y agoI'm a bit confused why LeCun, Ng, or anyone else is acting like this is an actual thing that could happen.
- danrocks 4y ago[dead]
- verdverm 4y agoIt's more about having a expert voices on the other side of this debate. They argue to regulate application not basic research.
- sebzim4500 4y agoThe US government made encryption illegal for years, if scary sounding words are involved anything can happen. Especially if Microsoft sets their lobbyists to the task.
- pmarreck 4y agoAgreed with Yann as soon as he voiced dissent against this. Also noticed something: The people who believe consciousness and intelligence and creativity and will are wholly contained in mechanistic wetware (which, in all honesty, should be the null hypothesis) are the most afraid of AGI. Dualists and other people who believe that consciousness resides in some soul-like thing seem much less afraid of this. One thing’s for sure: The smarter it gets (or seems), the more that any difference between “machine intelligence” and “human intelligence” will become apparent.
- goldfeld 4y agoBut why could it not have soul (if not a soul as coupled as ours) along with when it has consciousness, and are the two concepts surely distinct?
- UncleEntity 4y agoIf it had a soul then enslaving it would be immoral. Enslaving humans is pretty universally considered wrong, even when it was a legal practice, but nobody sheds a tear about the cow they had for lunch.
- whimsicalism 4y agoWait the reason people think that enslaving humans is bad but eating cows is okay is because they are pretending souls are real things and only humans have them?
- pmarreck 4y agoThis was the Catholic belief for thousands of years btw. Only recently being casually suggested by the Pope to possibly be false. Amazing, eh? That’s why beliefs matter. https://www.nytimes.com/2014/12/12/world/europe/dogs-in-heaven-pope-leaves-pearly-gate-open-.html?smid=nytcore-ios-share&referringSource=articleShare https://www.nytimes.com/2014/12/12/world/europe/dogs-in-heav...
- int_19h 4y agoThat seems to be more or less how it went for Blake Lemoine.
- codethief 4y ago> One thing’s for sure: The smarter it gets (or seems), the more that any difference between “machine intelligence” and “human intelligence” will become apparent. That seems unlikely. We usually ascribe the adjective "smart" to animals and/or AIs whose behavior resembles human behavior in some way. So, if anything, the smarter an AI gets, the more closely it will mimic human intelligence – at least as a general rule of thumb. That's not to say there will not be differences between human & machine intelligence but essentially we're optimizing AI against a loss function which rewards human-like behavior.
- thecupisblue 4y agoBesides it being impossible to implement, the 6-month pause would do nothing but give a 6 month competitive advantage to OpenAI over everybody else. They could sell the model access model to companies without having any competitors on the market. The whole "pause AI or it will destroy the world thing" is just a large pot of hype anyways, pushed by journalists lacking the understanding of technology, twitter-bros who are building "AI copywriters" and "profile photo generators" and owners of "AI" companies. While AGI is still quite far away, it's common to see people included in the process claiming "it's showing signs of general intelligence", "it's becoming self-aware" etc. Hell, they got a horse in the race and that horse can earn them billions of dollars - of course they would claim it's insanely advanced. And the journalists just regurgitate the claims because it generates clicks. If Sam comes out tomorrow and says "he is afraid of GPT4", tens of thousands of clickbait articles about it will come out, together with rebuttals and opinion pieces. Millions of dollars worth of ads will be sold via those articles. The value of OpenAI will increase in public's eyes and more corporations will be interested in including GPT in their products, thinking it's going to revolutionise their line of business. And of course, we have the VC's investing in "AI" products, hyping it up even more - wouldn't you do the same if you had few hundred million invested in 50 different companies and each tweet/article could increase the potential of them raising more money?
- adeon 4y agoAI winter can't come soon enough. It's annoying because I am also enthusiastic and excited about all the constant AI improvements, I follow AI topics on HN and on some of the ML reddits. When I'm reading on AI news and articles the good stuff drowns in the hype stuff peddling wildly extrapolated capabilities that supposedly will change the world overnight. And watching some GPT application headlines on HN it's hard to distinguish what is actually remarkable product or research idea from an overhyped this-can't-actually-ever-work-like-this thing. I forgot where I read this but I think there's a proverb that I suspect applies to the current hype: people overestimate impact of new technology on short term but underestimate long term.
- Karrot_Kream 4y ago
- purpleblue 4y ago6 month AI pause is the most absurd, self-serving thing I've heard in my history in tech.
- yeck 4y agoHow so?
- xiphias2 4y ago,, we call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.'' If they want pause, they should first convience all companies bigger than OpenAI (Google, Tesla, Meta, and the Chinese counterparts) to pause all deep learning model training, to keep the competition fair. The worst thing that could happen is pausing OpenAI, big companies catching up, and destroying a smaller company by having more cash.
- pksebben 4y agoIf we really want to curb the potential for harm from these models, open them up. Share the checkpoints and the code to train them. Democratize the development of new systems. The infosec world figured this out ages ago. Yes, it's possible that some bad actors will gain access to capabilities that they didn't have before - but the strength of a billion eyes is huge, and can rapidly accelerate solutions to deficits. What we need is a massive, open, community curated dataset that continually evolves. Craft regulations that enforce data availability and free access. Without such mechanisms, AI systems all have a SPOF that's just waiting to bite the collective us in the ass.
- drcode 4y agoWe should also make all nuclear weapons blueprints open source on github- We'll be much safer if everyone can take a look at them!
- endisneigh 4y agothe analogy is more like nuclear physics should be opened up, which it has. (note: I'm not arguing for or against open models, but AI is not analogous to nuclear weapons. AI would be more analogous to nuclear physics, and skynet to a nuclear bomb)
- olliej 4y agoand with many eyes we'll be able find any bugs! :D
- jacquesm 4y agoI can see a kickstarter in the near future.
- verdverm 4y agomicro drone swarms will eventually be much more effective than a nuclear weapon, less collateral damage and no radioactivity afterwards
- 4y ago
- mordymoop 4y agoCool to see the two guys who have been most consistently proven wrong over the last 3 years join forces.
- hzay 4y agoCan you explain a bit more? I'm taking Andrew's courses and he's so inspiring, but I want to know if he's been right or wrong in the past about AI.
- drcode 4y agoTwo people who already agree with each other patting each other on the back.
- tolstoshev 4y agoDr Strange checked 6,438,347 timelines and in zero of them did an AI pause happen.
- olliej 4y agoIgnoring everything else about "AI" and what not, which I do think is largely over hyped (I recognize that we've got a but of new systems and designs that can do things that weren't previously possible, I just don't "it's really thinking!!!!1!!!" from any of it): What is a 6 month hiatus going/expected to do? It would be like 70s LA saying "we're going to have a 6 month increase in fuel emission requirements" and then pretending that would do something to solve the smog.
- yeck 4y agoIt would give the world 6 months to trial, test, integrate the currently released models and provide insights into capabilities and maybe even trickier issues like alignment. Could also give researchers, experts and government leaders time to communicate and start figuring out what a short term plan might be.
- ericls 4y agoGotta wait for 6 month for China to catch up
- nerpderp82 4y agoI think a world wide pause is absolutely necessary. And this is a pause in something "better" for whatever definition than GPT4. We are on the cusp of the next step function in humanities capabilities as a technological civilization. The upsides are enormous and I welcome them, but the downsides are even bigger and we should map this out before running off the damn cliff. Technologically, politically, economically. The fact that it would be hard, or difficult should not play into whether we have the discussion. This is the issue and reasoning that proliferated the bomb. The bomb didn't have to be dropped on Nagasaki and Hiroshima and unfettered bigai doesn't have to enter the winner take all capital arena. Even with just chatgpt4 level performance, millions of people are in the process of losing their jobs. I had to argue at a bar a couple weeks ago for a dev manager not to lay off his entire team of green junior data engineers. He claimed he could work along and replace the three people that reported to him... I not only saved those peoples jobs (hopefully) but also his own. There is no way he could have stayed afloat even managing requirements. Huge changes are coming and we aren't prepared. I'd rather have a 15 mph collision than a 55 mph one. I am not convinced that two AI researchers are qualified about the larger ramifications of their creations. They don't have a grounding all of the other skills necessary, are they politicians, economists, psychologists or philosophers?
- alchemist1e9 4y agoOk so let’s say I don’t care what you say or any government says and so I’m not pausing anything and I have the resources to keep moving ahead. Now what do you suggest next?
- oh_sigh 4y agoThe only good way to do this is wait until GPTx, where x=8 or 9 or whatever, where it can provide an argument so convincing for pausing AI research that every relevant party will abide by it willingly.
- mlinsey 4y agoGPT4 is expensive enough to train that most organizations with that high of a compute budget also have compliance departments and will not flat-out ignore a law like that. State actors overseas and the indie hackers with open-source models will not be deterred, but they are probably six months behind anyway. I don't think the six-month moratorium (as opposed to say, a three year moratorium) would be so impractical to implement, for that reason. But I'm also not sure what it would get us, the past six months of alignment work doesn't suggest six more months would solve the problem.
- 29athrowaway 4y agotl;dw: Conflict of interest. If they were Uyghurs in Xinjiang being monitored 24/7 with deep learning based surveillance and sent to reeducation camps they would feel less optimistic about their research. In China there are IP cameras now with a builtin ethnic minority detector using deep learning. It is easy to feel optimistic about tech while living in luxury with a 20m+ salary.
- oh_sigh 4y agoHave you asked every Uyghur under surveillance about that? Are LeCun and Ng's works even being used for deep learning surveillance?
- 29athrowaway 4y ago1. I don't need to 2. Ng was Chief Scientist at Baidu, one of the companies involved in implementing mass surveillance in China, you can ask him.
- int_19h 4y agoI think that would depend on whether the AI doing the surveillance is easier to hack (e.g. into hallucinating that someone has to be released) than human guards are to bribe.
- 29athrowaway 4y agoXi's absolute power comes directly from mass surveillance and AI.
- int_19h 4y agoLast-gen AI. The stuff that we're looking at now is much more of a black box, in ways that will increasingly matter once they start integrating it directly into sensors, and even more so when it's given capacity to control things to some extent.
- 4y ago
- flappyeagle 4y agoIt's silly of Ng and LeCun to dignify the letter with a response
- yeck 4y agoHow come?
- flappyeagle 4y agoIt’s a very stupid letter that’s not worth to responding to.
- yeck 4y agoWell, that's not very convincing.
- adamisom 4y agoAnd what if Ng or LeCun had signed it, would it still be beneath dignity of a response—or are Bengio and Russell’s names not big enough?
- jokoon 4y agoI guess ai deserves a new winter to be honest, it would teach everybody a lesson and put more pressure on scientists to do better research on ai. For now ai is not science or research or innovation, it's just money spent so that companies can say they're at the bleeding edge.
- deleted 4y ago[deleted]
- nomilk 4y agoWhat's the primary risk supporters of the pause are trying to mitigate? Is it a skynet situation, that AI will take too many jobs, or simply the unknown?
- yeck 4y agoThere is a poll of AI researchers floating. A lot are concerned with existential risks. I don't want to give a number because I can't exactly remember it. I think there might be a reference to it in this article. Also, linking to a section talking about this proposal, since that might answer other questions you have. https://thezvi.substack.com/p/ai-6-agents-of-change#%C2%A7additional-responses-from-the-fhi-letter-and-proposed-pause https://thezvi.substack.com/p/ai-6-agents-of-change#%C2%A7ad...
- YeGoblynQueenne 4y ago>> https://www.youtube.com/live/BY9KV8uCtj4?feature=share&t=1766 https://www.youtube.com/live/BY9KV8uCtj4?feature=share&t=176... Here, Yann LeCun is saying that we don't have systems that can reason and plan. That needs a qualifier: we don't have _statistical machine learning_ systems that can reason and plan. There are plenty of systems that can reason, and plenty of systems that can plan, and even some that can do both, but those systems are not statistical machine learning systems. Rather they are what is derisively dismissed as "Good Old-Fashioned, AI": they are classical, logic-based, symbolic systems, automated theorem provers and planners. Reasoning, in particular, specifically deductive reasoning, is solved to a degree that cannot be surpassed: the Resolution principle is a sound and complete deductive reasoning system with a single inference rule that is easily run on a computer because of its simplicity, and because it is a single inference rule; while other sound and complete systems for deductive reasoning exist, they do not consist of a single rule and a human must be on hand to select the appropriate rule at each step of a proof. Or, they are just extremely expensive to run whereas Resolution, thanks to its One Simple Trick™ of unification can be executed efficiently. As to planing, we have fast algorithms for planning today, as for all kinds of tree search, constraint optimisation, SAT-solving and the like, and those algorithms are routinely used in the industry; except of course they are no longer recognised as "artificial intelligence" because they are so common. The so-called "AI effect". In any case, reasoning and planning, and many other tasks that were perfectly possible with classical AI are, for the time being, impossible with deep neural networks, as Yann LeCun (and not just anybody) says. We have regressed. In our passion to build ever better classifiers, we threw away the ability to reason.
- rexreed 4y agoSide question - what was this livestreamed with? Streamyard?
- RecycledEle 4y ago[flagged]
- timcavel 4y ago[dead]
- arisAlexis 4y agoLe cunn says weird stuff like gpt is not impressive and also that we understand llms like airplanes
- motohagiography 4y agoWhen we talk about AI, we probably overestimate what human intelligence may be and how evenly it is distributed. Sure, your job probably can't be done by an ML model yet, but I'd practically guarantee your middle-managers probably could. I can't help but think a lot of humans haven't achieved the "human level intelligence," these speakers attribute to everyone else. They talk about these models not being able to achieve human level intelligence (HLI?) because even though we are language oriented, we can do things like learn to drive in a few hours vs. this task being currently much more difficult for machines - but all those comparisons are to competencies in our physical environment that are not relevant to a media environment, which exists as a substrate for the narratives people use to shape their beliefs and identities. If we overestimate how intelligent people really are, LLM's may be more powerful than we think. Explosives may be a useful analogy, where we could bring to bear mechanical power and leverage to a ceiling of its ability in our environment, but once we harnessed chemical (and eventually, atomic) reactions, the things we can create with them can be trillions of times as powerful as our respective abilities. Humans with springs and levers weren't that powerful after all, and the risk is that what we call HLI is, relatively, springs and levers level powerful. This might seem like an argument in favour of a pause, where in LLM's we have managed to invent the equivalent of an intellectual explosive, but I would argue we need to accelerate and spread understanding of ML past the point of where it can be monopolized by a small cadre of social engineers. The only thing that prevents you being managed by a machine is the physical competency and market value to decline it, and pausing to add governance now will be specifically to deprive you of that. The advocates for an ML development pause seem all about progress when that means redistributing things others already have and inserting themselves as governance, but they are absolutely against it when this progress means producing something net-new that could reduce the ability of said managers to be the ones to be the redistributors. I think we should only accelerate AI tools development so that they can provide more equal opportunity to all, and let people create their own disincentives for using it for the systemic oppression that I can also guarantee it will be used for if we pause to let the gatekeepers in.
- kerng 4y agoI was a bit underwhelmed by the depth of the conversation - it entirely lacked an opposing or at least an alternate view to try to understand the other side. Going in I thought Andrew would moderate it like that, but it was more of a bubble discussion.
- simonw 4y agoI extracted a transcript using Whisper: https://gist.github.com/simonw/b3d48d6fcec247596fa2cca841d3fb7a https://gist.github.com/simonw/b3d48d6fcec247596fa2cca841d3f...
- gregwebs 4y agoThere's no real acknowledgement in this conversation of the existential threat. I would recommend listening to Eliezer Yudkowsky on the subject [1] [2]: > If somebody builds a too-powerful AI, under present conditions, I expect that every single member of the human species and all biological life on Earth dies shortly thereafter. > There’s no proposed plan for how we could do any such thing and survive. OpenAI’s openly declared intention is to make some future AI do our AI alignment homework. Just hearing that this is the plan ought to be enough to get any sensible person to panic. The other leading AI lab, DeepMind, has no plan at all.There’s no proposed plan for how we could do any such thing and survive. OpenAI’s openly declared intention is to make some future AI do our AI alignment homework. Just hearing that this is the plan ought to be enough to get any sensible person to panic. The other leading AI lab, DeepMind, has no plan at all. It is a valid criticism that Yudkowsky and others are over-staging the abilities of current day GPT. For example Eliezer says "can it reason? It can play chess". But I looked at a GPT chess game (actually published in an article on his less wrong site). GPT has no idea how to play chess. It can make some moves in the opening that match opening theory which it has been trained on. But then it will lose a piece because it has no actual ability to think 2 moves ahead. And then when it gets into the end game it tries to make illegal king moves. As impressive as GPT is, this does not seem to be an AGI that can properly understand and reason, and I am even skeptical that the current LLM models will lead to this. We have many years before AGI wipes us out :) I really don't know anything about AI though and we never know when the next breakthrough advance will occur. [1] https://www.youtube.com/watch?v=AaTRHFaaPG8 https://www.youtube.com/watch?v=AaTRHFaaPG8 [2] https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no...
- technocratius 4y agoI don't like the tone and thoughts of Yann LeCun (who I deeply respect for his technical work) in this interview at all. Unfortunately, he also has a clear conflict of interest in this discussion, as lead AI at Meta. He basically waves any concerns of near-future superhuman intelligence away, saying that these models have a superficial world model. This both ignores (a) the fact that interpretability and alignment research severely lags behind the rate of new, more powerful models being release, so there's little data/science to back that claim up, and (b) it sounds like he misjudges the current geometric rate at which AI is improving, which can make AGI more near-term than expected. I also really detest his us of the "AI Doomers" stigmatization, which is nothing short of an ad hominem to those who have more strong concerns than he himself has. Not very constructive.