8 ms·
Interview with Yann LeCun on AI
- singingwolfboy 2y agohttps://archive.ph/ooEEY https://archive.ph/ooEEY
- tikkun 2y agoIt's frustrating how many disagreements come down to framings rather than actual substance. His framing of intelligence is one thing. The people who disagree with him are framing intelligence a different way. End of story. I wish that all the energy went towards substantive disagreements rather than disagreements that are mostly (not entirely) rooted in semantics and definitions.
- jcranmer 2y agoThat's not what he's saying at all, though. What he's saying is that he thinks the current techniques for AI (e.g., LLMs) are near the limits of what you can achieve with such techniques and are thus a dead-end for future research; also consequently, hyperventilation about AI superintelligence and the like is extremely irresponsible. It's actually a substantial critique of AI today in its actual details, albeit one modulated by popular press reporting that's dumbing it down for popular consumption.
- aithrowawaycomm 2y agoHis point is that lots of AI folks are framing intelligence incorrectly by overvaluing surface knowledge or ability to be trained to solve constrained problems, when cats have deeper cognitive abilities like planning and rich spatial reasoning which are far beyond the reach of any AI in 2024. ANNs are extremely useful tools because they can process all sorts of information humans find useful: unlike animals or humans, ANNs don't have their own will, don't get bored or frustrated, and can focus on whatever you point them at. But in terms of core cognitive abilities - not surface knowledge, not impressive tricks, and certainly not LLM benchmarks - it is hard to say ANNs are smarter than a spider. (In fact they seem dumber than jumping spiders, which are able to form novel navigational plans in completely unfamiliar manmade environments. Even web-spinning spiders have no trouble spinning their webs in cluttered garages or pantries; would a transformer ANN be able to do that if it was trained on bushes and trees?)
- qarl 2y agoWell... except... cats can't talk.
- Scrapemist 2y agoMaybe AI can translate
- qarl 2y agoHEH... reminds me of an argument Searle once made: with the right "translation" you can make a wall intelligent.
- Barrin92 2y agoAnd as Marvin Minsky taught us, which is probably one of the most profound insights in the entire field, talking seems like an accomplishment to us because it's the least developed part of our capacities. It's so conscious a task not because it's a sign of intellect but because it's the least developed and most novel thing our brains do, which is why it's also likely the fastest to learn for a machine. Moving as smoothly as a cat and navigating the world is the part that actually took our brains millions of years to learn, and movement is effortless not because it's easy but because it took so long to master, so it's also going to be the most difficult thing to teach a machine. The cognitive stuff is the dumb part, and that's why we have chess engines, pocket calculators and chatbots before we have emotional machines, artificial plumbers and robots that move like spiders.
- qarl 2y agoI'm not sure that's the right way to look at it. Ten years ago, it was common to hear the argument: "Are cats intelligent? No, they can't speak." Language was seen as the pinnacle of the mind. Lately that's been flipped on its head, but only because machines have gotten so good at it. I think the real reason we don't have robots that move like spiders is that robots don't have muscles, and motors are a very poor approximation.
- 2y ago
- krig 2y ago(reacting to the title alone since the article is paywalled) AI can’t push a houseplant off a shelf, so there’s that. Talking about intelligence as a completely disembodied concept seems meaningless. What does ”cat” even mean if comparing to something that doesn’t have a physical corporeal presence in time and space. To compare like this seems to me like making a fundamental category error. edit: Quoting, “You’re going to have to pardon my French, but that’s complete B.S.” I guess I’m just agreeing with LeCun here.
- jcranmer 2y agoIt's referring to the fact that cats are able to do tasks like multistage planning, which he asserts current AIs are unable to do.
- krig 2y agoThanks, that makes more sense than the title. :)
- dang 2y agoWe replaced the baity title with something suitably bland. If there's a representative phrase from the article itself that's neutral enough, we could use that instead.
- qarl 2y ago> It's referring to the fact that cats are able to do tasks like multistage planning, which he asserts current AIs are unable to do. I don't understand this criticism at all. If I go over to ChatGPT and say "From the perspective of a cat, create a multistage plan to push a houseplant off a shelf" it will satisfy my request perfectly.
- staticman2 2y agoChatGPT only decides what to write one token at a time.
- mmoustafa 2y agoIt’s really hard for me to believe Yann is engaging sincerely, he is downplaying LLM abilities on purpose. He leads AI at Meta, a company with the competitive strategy to commoditize AI via Open Source models. Their biggest hinderance would be regulation putting a stop to the proliferation of capabilities. So they have to understate the power of the models. This is the only way Meta can continue sucking steam out of the leading labs.
- muglug 2y agoYou’re free to concoct a conspiracy that he’s just a puppet for Meta’s supposed business interests*, but that doesn’t change the validity of his claims. * pretty sure any revenue from commercial Llama licenses are a rounding error at best
- threeseed 2y ago> commoditize AI via Open Source models Sounds like we should be fully supporting them then.
- mmoustafa 2y agoYes!
- lyu07282 2y agoYou don't have to assume malice, he is a strong believer in liberalism so naturally he would argue whatever leads to less regulation. Even if he thought AI was dangerous he would still believe that corporations are better suited to combat that threat than any government. Its similar to how the WSJ journalist would never ask him what he thinks about the larger effects of the "deindustrialization" of knowledge-based jobs caused by AI. Not because the journalist is malicious, its just the shared, subconscious ideology. People don't need a reason to protect capital interests, even poor people on the very bottom will protect it.
- mmoustafa 2y agoYes, you can be liberalist and anti-regulation while also believing the capabilities are ... what they are. I believe many of Yann's statements are straw-manning or sometimes, simply incorrect downplaying of capabilities.
- deleted 2y ago[deleted]
- mrandish 2y agoWhile I'm no expert on AI, everything I've read from LeCunn on AI risk so far strikes me as directionally correct. I keep revisting the best examples I can find of the 'Foom' hypothesis and it just doesn't seem likely. Not to say that AI won't be both very useful and disruptive, just that existential fears like Skynet scenarios don't strike me as plausible.
- Elucalidavah 2y ago> it just doesn't seem likely It is likely conditional on the price of compute dropping the way it has been. If you can basically simulate a human brain on a $1000 machine, you don't really need to employ any AI researchers. Of course, there has been some fear that the current models are a year away from FOOMing, but that does seem to be just the hype talking.
- threeseed 2y agoIf you can simulate a human brain and it required a $100b machine you would still get funding in a weekend. Because you could easily find ways to print money e.g. curing types of cancers or inventing a better Ozempic. But the fact is that there is no path to simulating a human brain.
- llamaimperative 2y agoThere is no path to it? That's a bold claim. Are brains imbued with special brain-magic that makes them more than, at rock bottom, a bunch of bog-standard chemical and electrical and thermal reactions? It seems very obviously fundamentally solvable, though I agree it is nowhere in the near future.
- bob1029 2y ago> Are brains imbued with special brain-magic that makes them more than, at rock bottom, a bunch of bog-standard chemical and electrical and thermal reactions? Some have made this argument (quantum effects, external fields, etc.). If any of these are proven to be true then we are looking at a completely different roadmap.
- jimjimjim 2y agoLLMs are great! They are just not what I would call Intelligence
- miohtama 2y agoIf you don't call it intelligence you miss the enormous political and social opportunity to go down in the history as the pioneer of AI regulation (:
- razodactyl 2y agoHe is highly knowledgeable in his field. He's very abrasive in his conduct but don't mistake it for incompetence. Even the "AI can't do video" thing was blown out and misquoted because discrediting people and causing controversy fuels more engagement. He actually said something along the lines of it "not being able to do it properly" / everything he argues is valid from a scientific perspective. The joint embeddings work he keeps professing has merit. --- I think the real problem is that from a consumer perspective, if the model can answer all their questions it must be intelligent / from a scientist's perspective it's not capable for the set of all consumers so it's not intelligent. So we end up with a dual perspective where both are correct due to technical miscommunication and misunderstanding.
- kergonath 2y ago> He is highly knowledgeable in his field Indeed. It seems to me that he has a type of personality common in skilled engineers. He is competent and makes informed decisions, but does not necessarily explain them well (or at all if they feel trivial enough), is certain of his logic (which often sounds like arrogance), and does not seem to have much patience. He is great at what he does but he really is not a spokesman. His technical insights are often very interesting, though.
- kordlessagain 2y agoTechnical gatekeeping is a thing, and definitely a problem at times. This is particularly problematic when dealing with concepts that aren't fully understood, even by their originators, as these might contain valuable insights despite their apparent strangeness. The risk of losing out on potentially groundbreaking perspectives must be weighed against the importance of scientific integrity. To address this, the technical community could benefit from fostering more open dialogue, encouraging interdisciplinary collaboration, and creating spaces where speculative ideas can be explored without immediate judgment. Usually doesn't happen here.
- zer0c00ler 2y ago[dead]
- Yacovlewis 2y agoFrom my own experience trying to build an intelligent digital twin startup based on the breakthrough in LLM's, I agree with LeCunn that LLMs are actually quite far from demonstrating the intelligence of house cats, and I myself likely jumped the gun by trying to emulate intelligent humans with the current stage of AI. His AI predictions remind me of Prof. Rodney Brooks (MIT, Roomba) and his similarly cautious timelines for AI development. Brooks has a very strong track record over decades of being pretty accurate with his timelines.
- steveBK123 2y agoI would suspect any possible future AGI-like progress would be some sort of ensemble. LLMs may be a piece of the puzzle, but they aren't a single model to AGI solution.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- twobitshifter 2y agoThe recent advance of reasoning in o1 preview seems to have not been widely understood by the media and even LeCun in this case. o1 preview represents a new training paradigm using reinforcement learning on the reasoning steps applied to the solution. This step allows for reasoning to be developed just like AlphaZero is able to ‘reason’ and come up with unique solutions. The reinforcement learning in o1preview means that the ‘repeating facts learned’ arguments no longer apply. Instead the AI is free to come up with its own reasoning steps that lead to correct answers and these reasoning steps are refined over time. It can continue to train and get better by repeatedly answering the same questions, the same way alphazero can play the same game multiple times.
- david-gpu 2y agoI don't think it is comparable to alpha zero, because in the latter case there is an easily verifiable endpoint: whether it won the match or not. With o1 they still rely on human-curated "correct" answers; the system is not able to decide by itself what a correct answer is like alpha zero does. Thus, while it leads to improved reasoning vs prior art, it is not yet bootstrapping. It may be useful in constrained fields like coming up with proofs of theorems, though.
- maskil 2y agoYou still will need clear benchmarks as the reward for RL. With Chess, the rules are simple but you may not have a clear loss function for a complicated architectural challenge.
- Reubend 2y agoI don't think that's correct. As I understand it, O1 can use reinforcement learning to optimize its chain of thought, but each individual "thought" is still subject to the performance of the LLM. Therefore, while it can generate new strategies to approach a problem, the implementation of each step within the strategy is limited by a probabilistic approach. Contrast that with AlphaZero, which can come up with strategies that are 100% unique (in theory), since it isn't constrained by any human training. I think O1 is a step forward, but not a massive leap in technology.
- 2y ago
- ks2048 2y agoIt seems the problem with this "debate" is that intelligence is very clearly not a scalar value - where you can look at a number line and put a dot where cats are, where humans are, where AIs are, etc. So, people just talk past each other, with everyone using a different method on how a complex trait like "intelligence" is to be collapsed down to a scalar for easy comparison.
- echelon 2y agoWe don't understand intelligence. But we do understand vision and hearing. We have for over 50 years. We've implemented them classically and described them with physics. Game engines, graphics pipelines, synthesizers, codecs, compression, digital image processing, ... the field is vast and productive. Our mastery over signals is why I'm so bullish on diffusion and AI for images, video, and audio regardless with whatever happens with LLMs. And if this tech cycle only improves our audio-visual experience and makes games and film more accessible, it'll still be revolutionary and a step function improvement over what came before.
- deepfriedchokes 2y agoJudging fish by their ability to climb trees.
- OKRainbowKid 2y agoThat's a good analogy for when people use the task of counting "r"s in "strawberry" to make a point that LLMs are useless/dumb.
- yunohn 2y agoI mean, OpenAI made this the defining purpose of their o1 “reasoning” model, and teased it for weeks with a strawberry across Twitter.
- blackeyeblitzar 2y agoI feel like LeCun does the same interview and the same presentation over and over. He’s obsessed with the cat analogy and the notion that JEPA will succeed transformer based ‘traditional’ LLM architecture. Maybe that is true but I feel like he has too absolute a view on these topics. Sometimes I feel like I am listening to a politician or marketer rather than someone making forward progress in the field.
- benlivengood 2y agoI think I would put more credence in Yann LeCun's predictions if he had predicted the emergence of few-shot learning and chain-of-thought reasoning as a function of model size/data before they happened. In part, he's arguing that LLMs are not the most efficient path toward intelligence and that some other design will do better, which is probably true, but no one has pushed model size for the (somewhat ironically named) Gato-style multimodal embodied transformers that I think would result in something closer to cat intelligence. I am reasonably certain that further step changes will happen as LLM model and/or data sizes increase. Right now we're achieving a lot of SOTA performance with somewhat smaller models and multimodal pretraining, but not putting those same methods to work in training even larger models on more data.
- arisAlexis 2y agoAre cars still smarter than o1 or he did a somersault