12 ms·

Ask HN: Any insider takes on Yann LeCun's push against current architectures?

• ActorNightly 2y ago
  Not an official ML researcher, but I do happen to understand this stuff. The problem with LLMs is that the output is inherently stochastic - i.e there isn't a "I don't have enough information" option. This is due to the fact that LLMs are basically just giant look up maps with interpolation. Energy minimization is more of an abstract approach to where you can use architectures that don't rely on things like differentiability. True AI won't be solely feedforward architectures like current LLMs. To give an answer, they will basically determine alogrithm on the fly that includes computation and search. To learn that algorithm (or algorithm parameters), at training time, you need something that doesn't rely on continuous values, but still converges to the right answer. So instead you assign a fitness score, like memory use or compute cycles, and differentiate based on that. This is basically how search works with genetic algorithms or PSO.
  • seanhunter 2y ago
    > The problem with LLMs is that the output is inherently stochastic - i.e there isn't a "I don't have enough information" option. This is due to the fact that LLMs are basically just giant look up maps with interpolation. I don't think this explanation is correct. The input to the decoder at the end of all the attention heads etc (as I understand it) is a probability distribution over tokens. So the model as a whole does have an ability to score low confidence in something by assigning it a low probability. The problem is that thing is a token (part of a word). So the LLM can say "I don't have enough information" to decide on the next part of a word but has no ability to say "I don't know what on earth I'm talking about" (in general - not associated with a particular token).
    • estebarb 2y ago
      The problem is exactly that: the probability distribution. The network has no way to say: 0% everyone, this is non sense, backtrack everything. Other architectures, like energy based models or bayesian ones can assess uncertainty. Transformers simply cannot do it (yet). Yes, there are ways to do it, but we are already spending millions to get coherent phrases, few ones will burn billions to train a model that can do that kind of assessments.
      • ortsa 2y ago
        Has anybody ever messed with adding a "backspace" token?
        • refulgentis 2y ago
          Yes. (https://news.ycombinator.com/item?id=36425375 https://news.ycombinator.com/item?id=36425375, believe there's been more) There's a quite intense backlog of new stuff that hasn't made it to prod. (I would have told you in 2023 that we would have ex. switched to Mamba-like architectures in at least one leading model) Broadly, it's probably unhelpful that: - absolutely no one wants the PR of releasing a model that isn't competitive with the latest peers - absolutely everyone wants to release an incremental improvement, yesterday - Entities with no PR constraint, and no revenue repurcussions when reallocating funds from surely-productive to experimental, don't show a significant improvement in results for the new things they try (I'm thinking of ex. Allen Institute) Another odd property I can't quite wrap my head around is the battlefield is littered with corpses that eval okay-ish, and should have OOM increases in some areas (I'm thinking of RWKV, and how it should be faster at inference), and they're not really in the conversation either. Makes me think either A) I'm getting old and don't really understand ML from a technical perspective anyway or B) hey, I 've been maintaining a llama.cpp wrapper that works on every platform for a year now, I should trust my instincts: the real story is UX is king and none of these things actually improve the experience of a user even if benchmarks are ~=.
          • vessenes 2y ago
  • throw310822 2y ago
    > there isn't a "I don't have enough information" option. This is due to the fact that LLMs are basically just giant look up maps with interpolation. Have you ever tried telling ChatGPT that you're "in the city centre" and asking it if you need to turn left or right to reach some landmark? It will not answer with the average of the directions given to everybody who asked the question before, it will answer asking you to tell it where you are precisely and which way you are facing.
    • wavemode 2y ago
      That's because, based on the training data, the most likely response to asking for directions is to clarify exactly where you are and what you see. But if you ask it in terms of a knowledge test ("I'm at the corner of 1st and 2nd, what public park am I standing next to?") a model lacking web search capabilities will confidently hallucinate (unless it's a well-known park). In fact, my person opinion is that, therein lies the most realistic way to reduce hallucination rates: rather than trying to train models to say "I don't know" (which is not really a trainable thing - models are fundamentally unaware of the limits of their own training data), instead just train them on which kinds of questions warrant a web search and which ones should be answered creatively.
      • QuesnayJr 2y ago
        I tried this just now on Chatbot Arena, and both chatbots asked for more information. One was GPT 4.5 preview, and one was cohort-chowder (which is someone's idea of a cute code name, I assume).
        • wavemode 2y ago
          I tried this just now on Chatbot Arena, and both chatbots very confidently got the name of the park wrong. Perhaps you thought I meant "1st and 2nd" literally? I was just using those as an example so I don't reveal where I live. You should use actual street names that are near a public park, and you can feel free to specify the city and state.
  • josh-sematic 2y ago
    I don’t buy Lecun’s argument. Once you get good RL going (as we are now seeing with reasoning models) you can give the model a reward function that rewards a correct answer most highly, an “I’m sorry but I don’t know” less highly than that, a wrong answer penalized, a confidently wrong answer more severely penalized. As the RL learns to maximize rewards I would think it would find the strategy of saying it doesn’t know in cases where it can’t find an answer it deems to have a high probability of correctness.
    • Tryk 2y ago
      How do you define the "correct" answer?
      • jpadkins 2y ago
        obviously the truth is what is the most popular. /s
      • josh-sematic 2y ago
        Certainly not possible in all domains but equally certainly possible in some. There’s not much controversy about the height of the Eiffel Tower or how to concatenate two numpy arrays.
  • thijson 2y ago
    I watched an Andrej Karpathy video recently. He said that hallucination was because in the training data there were no examples where the answer is, "I don't know". Maybe I'm misinterpreting what he was saying though. https://www.youtube.com/watch?v=7xTGNNLPyMI&t=4832s https://www.youtube.com/watch?v=7xTGNNLPyMI&t=4832s
  • TZubiri 2y ago
    If multiple answers are equally likely, couldn't that be considered uncertainty? Conversely if there's only one answer and there's a huge leap to the second best, that's pretty certain.
  • unsupp0rted 2y ago
    > The problem with LLMs is that the output is inherently stochastic Isn't that true with humans too? There's some leap humans make, even as stochastic parrots, that lets us generate new knowledge.
    • borgdefenser 2y ago
      I think it is because we don't feel the random and chaotic nature of what we know as individuals. If I had been born a day earlier or later I would have a completely different life because of initial conditions and randomness but life doesn't feel that way even though I think this is obviously true.
  • spmurrayzzz 2y ago
    > i.e there isn't a "I don't have enough information" option. This is true in terms of default mode for LLMs, but there's a fair amount of research dedicated to the idea of training models to signal when they need grounding. SelfRAG is an interesting, early example of this [1]. The basic idea is that the model is trained to first decide whether retrieval/grounding is necessary and then, if so, after retrieval it outputs certain "reflection" tokens to decide whether a passage is relevant to answer a user query, whether the passage is supported (or requires further grounding), and whether the passage is useful. A score is calculated from the reflection tokens. The model then critiques itself further by generating a tree of candidate responses, and scoring them using a weighted sum of the score and the log probabilities of the generated candidate tokens. We can probably quibble about the loaded terms used here like "self-reflection", but the idea that models can be trained to know when they don't have enough information isn't pure fantasy today. [1] https://arxiv.org/abs/2310.11511 https://arxiv.org/abs/2310.11511 EDIT: I should also note that I generally do side with Lecun's stance on this, but not due to the "not enough information" canard. I think models learning from abstraction (i.e. JEPA, energy-based models) rather than memorization is the better path forward.
  • kjrfghslkdjfl 2y ago
    [dead]
  • itkovian_ 2y ago
    >This is due to the fact that LLMs are basically just giant look up maps with interpolation. This is obviously not true at this point except for the most loose definition of interpolation. >don't rely on things like differentiability. I've never heard lecun say we need to move away from gradient descent. The opposite actually.
  • throwawaymaths 2y ago
    i dont think the stochasticity that's the problem -- the problem is that model gets "locked in" once it picks a token and there's no takesies backsies. that also entails information destruction in the form of the logits table, but for the most part that should be accounted for in the last step before final feedforward
• stevenae 2y ago
  https://en.m.wikipedia.org/wiki/Energy-based_model https://en.m.wikipedia.org/wiki/Energy-based_model
• jawiggins 2y ago
  I'm not an ML researcher, but I do work in the field. My mental model of AI advancements is that of a step function with s-curves in each step [1]. Each time there is an algorithmic advancement, people quickly rush to apply it to both existing and new problems, demonstrating quick advancements. Then we tend to hit a kind of plateau for a number of years until the next algorithmic solution is found. Examples of steps include, AlexNet demonstrating superior image labeling, LeCun demonstrating DeepLearning, and now OpenAI demonstrating large transformer models. I think in the past, at each stage, people tend to think that the recent progress is a linear or exponential process that will continue forward. This lead to people thinking self driving cars were right around the corner after the introduction of DL in the 2010s, and super-intelligence is right around the corner now. I think at each stage, the cusp of the S-curve comes as we find where the model is good enough to be deployed, and where it isn't. Then companies tend to enter a holding pattern for a number of years getting diminishing returns from small improvements on their models, until the next algorithmic breakthrough is made. Right now I would guess that we are around 0.9 on the S curve, we can still improve the LLMs (as DeepSeek has shown wide MoE and o1/o3 have shown CoT), and it will take a few years for the best uses to be brought to market and popularized. As you mentioned, LeCun points out that LLMs have a hallucination problem built into their architecture, others have pointed out that LLMs have had shockingly few revelations and breakthroughs for something that has ingested more knowledge than any living human. I think future work on LLMs are likely to make some improvement on these things, but not much. I don't know what it will be, but a new algorithm will be needed to induce the next step on the curve of AI advancement. [1]: https://www.open.edu/openlearn/nature-environment/organisations-environmental-management-and-innovation/content-section-1.7 https://www.open.edu/openlearn/nature-environment/organisati...
  • Matthyze 2y ago
    > Each time there is an algorithmic advancement, people quickly rush to apply it to both existing and new problems, demonstrating quick advancements. Then we tend to hit a kind of plateau for a number of years until the next algorithmic solution is found. That seems to be how science works as a whole. Long periods of little progress between productive paradigm shifts.
• TrainedMonkey 2y ago
  This is a somewhat nihilistic take with an optimistic ending. I believe humans will never fix hallucinations. Amount of totally or partially untrue statements people make is significant. Especially in tech, it's rare for people to admit that they do not know something. And yet, despite all of that the progress keeps marching forward and maybe even accelerating.
  • ketzo 2y ago
    Yeah, I think a lot of people talk about "fixing hallucinations" as the end goal, rather than "LLMs providing value", which misses the forest for the trees; it's obviously already true that we don't need totally hallucination-free output to get value from these models.
    • mdp2021 2y ago
      Even as language models can partially solve a few problems, we remain with the problem of achieving Artificial General Intelligence, that the presence of LLMs has exacerbated because they so often reveal to be artificial morons. Intelligence finds solutions - actual, solid solutions. More than "fixing" hallucinations, the problem is going beyond them (arriving to "sobriety").
  • dtnewman 2y ago
    I’m not sure I follow. Sure, people lie, and make stuff up all the time. If an LLM goes and parrots that, then I would argue that it isn’t hallucinating. Hallucinating would be where it makes something up that is not in its training site nor logically deducible from it.
  • esafak 2y ago
    I think most humans are perfectly capable of admitting to themselves when they do not know something. Computers ought to do better.
    • danielmarkbruce 2y ago
      You must interact with a very different set of humans than most.
• deleted 2y ago
  [deleted]
• ALittleLight 2y ago
  I've never understood this critique. Models have the capability to say: "oh, I made a mistake here, let me change this" and that solves the issue, right? A little bit of engineering and fine tuning - you could imagine a model producing a sequence of statements, and reflecting on the sequence - updating things like "statement 7, modify: xzy to xyz"
  • fhd2 2y ago
    I get "oh, I made a mistake" quite frequently. Often enough, it's just another hallucination, just because I contested the result, or even just prompted "double check this". Statistically speaking, when someone in a conversation says this, the other party is likely to change their position, so that's what an LLM does, too, replicating a statistically plausible conversation. That often goes in circles, not getting anywhere near a better answer. Not an ML researcher, so I can't explain it. But I get a pretty clear sense that it's an inherent problem and don't see how it could be trained away.
  • rscho 2y ago
    "Oh, I emptied your bank account here, let me change this." For AI to really replace most workers like some people would like to see, there are plenty of situations where hallucinations are a complete no-go and need fixing.
  • croes 2y ago
    Isn’t that the answer if you tell them they are wrong?
• killthebuddha 2y ago
  I've always felt like the argument is super flimsy because "of course we can _in theory_ do error correction". I've never seen even a semi-rigorous argument that error correction is _theoretically_ impossible. Do you have a link to somewhere where such an argument is made?
  • aithrowawaycomm 2y ago
    In theory transformers are Turing-complete and LLMs can do anything computable. The more down-to-earth argument is that transformer LLMs aren't able to correct errors in a systematic way like Lecun is describing: it's task-specific "whack-a-mole," involving either tailored synthetic data or expensive RLHF. In particular, if you train an LLM to do Task A and Task B with acceptable accuracy, that does not guarantee it can combine the tasks in a common-sense way. "For each step of A, do B on the intermediate results" is a whole new Task C that likely needs to be fine-tuned. (This one actually does have some theoretical evidence coming from computational complexity, and it was the first thing I noticed in 2023 when testing chain-of-thought prompting. It's not that the LLM can't do Task C, it just takes extra training.)
  • vhantz 2y ago
    > of course we can _in theory_ do error correction Oh yeah? This is begging the question.
  • tyronehed 2y ago
    As soon as you need to start leaning heavily on error correction, that is an indication that your architecture and solution is not correct. The final solution will need to be elegant and very close to a perfect solution immediately. You must always keep close to the only known example we have of an intelligence which is the human brain. As soon as you start to wander away from the way the human brain does it, you are on your own and you are not relying on known examples of intelligence. Certainly that might be possible, but since there's only one known example in this universe of intelligence, it seems ridiculous to do anything but stick close to that example, which is the human brain.
• blueyes 2y ago
  Sincere question - why doesn't RL-based fine-tuning on top of LLMs solve this or at least push accuracy above a minimum acceptable threshhold in many use cases? OAI has a team doing this for enterprise clients. Several startups rolling out of current YC batch are doing versions of this.
  • InkCanon 2y ago
    If you mean the so called agentic AI, I don't think it's several. Iirc someone in the most recent demo day mentioned ~80%+ were AI
• __rito__ 2y ago
  Sligtly related: Energy Based Models (EBMs) are better in theory and yet too resource intensive. I tried to sell using EBMs to my org, but the price for even a small use case was prohibitive. I learned it from: https://youtube.com/playlist?list=PLLHTzKZzVU9eaEyErdV26ikyolxOsz6mq https://youtube.com/playlist?list=PLLHTzKZzVU9eaEyErdV26ikyo... Yann LeCun, and Michael Bronstein and his colleagues have some similarities in trying to properly Sciencify Deep Learning. Yann LeCun's approach, as least for Vision has one core tenet- energy minimization, just like in Physics. In his course, he also shows some current arch/algos to be special cases for EBMs. Yann believes that understanding the Whys of the behavior of DL algorithms are going to be beneficial in the long term rather than playing around with hyper-params. There is also a case for language being too low-dimensional to lead to AGI even if it is solved. Like, in a recent video, he said that the total amount of data existing on all digitized books and internet are the same as what a human children takes in in the first 4/5 years. He considers this low. There are also epistemological arguments against language not being able to lead to AGI, but I haven't heard him talk about them. He also believes that Vision is a more important aspect of intellgence. One reason being it being very high-dim. (Edit) Consider an example. Take 4 monochrome pixels. All pixels can range from 0 to 255. 4 pixels can create 256^4 = 2^32 combinations. 4 words can create 4! = 24 combinations. Solving language is easier and therefore low-stakes. Remember the monkey producing a Shakespeare play by randomly punching typewriter keys? If that was an astronomically big number, think how obscenely long it would take a monkey to paint Mona Lisa by randomly assigning pixel values. Left as an exercise to the reader. Juergen Schmidhuber has gone a lot queit now. But he also told that a world-model, explicitly included in training is reasoning is better, rather than only text or image or whatever. He has a good paper with Lucas Beyer.
  • vessenes 2y ago
    Thanks. This is interesting. What kind of equation is used to assess an ebm during training? I’m afraid I still don’t get the core concept well enough to have an intuition for it.
  • tyronehed 2y ago
    Since this exposes the answer, the new architecture has to be based on world model building.
• hnfong 2y ago
  I'm not an insider and I'm not sure whether this is directly related to "energy minimization", but "diffusion language models" have apparently gained some popularity in recent weeks. https://arxiv.org/abs/2502.09992 https://arxiv.org/abs/2502.09992 https://www.inceptionlabs.ai/news https://www.inceptionlabs.ai/news (these are results from two different teams/orgs) It sounds kind of like what you're describing, and nobody else has mentioned it yet, so take a look and see whether it's relevant.
  • hnuser123456 2y ago
    And they seem to be about 10x as fast as similar sized transformers.
    • 317070 2y ago
      No, 10x less sampling steps. Whether or not that means 10x faster remains to be seen, as a diffusion step tends to be more expensive than an autoregressive step.
      • littlestymaar 2y ago
        If I understood correctly, in practice they show actual speed improvement on high-end cards, because autoregressive LLMs are bandwidth limited and do not compute bound, so switching to a more expensive but less memory bandwidth heavy is going to work well on current hardware.
        • AlexCoventry 2y ago
          The SEDD architecture [1] probably allows for parallel sampling of all tokens in a block at once, which may be faster but not necessarily less computationally demanding in terms of runtime times computational resources used. [1] Which Inception Labs's new models may be based on; one of the cofounders is a co-author. See equations 18-20 in https://arxiv.org/abs/2310.16834 https://arxiv.org/abs/2310.16834
• estebarb 2y ago
  I have no idea about EBM, but I have researched a bit on the language modelling side. And let's be honest, GPT is not the best learner we can create right now (ourselves). GPT needs far more data and energy than a human, so clearly there is a better architecture somewhere waiting to be discovered. Attention works, yes. But it is not naturally plausible at all. We don't do quadratic comparisons across a whole book or need to see thousands of samples to understand. Personally I think that in the future recursive architectures and test time training will have a better chance long term than current full attention. Also, I think that OpenAI biggest contribution is demostrating that reasoning like behaviors can emerge from really good language modelling.
• probably_wrong 2y ago
  I haven't read Yann Lecun's take. Based on your description alone my first impression would be: there's a paper [1] arguing that "beam search enforces uniform information density in text, a property motivated by cognitive science". UID claims, in short, that a speaker only delivers as much content as they think the listener can take (no more, no less) and the paper claims that beam search enforced this property at generation time. The paper would be a strong argument against your point: if neural architectures are already constraining the amount of information that a text generation system delivers the same way a human (allegedly) does, then I don't see which "energy" measure one could take that could perform any better. Then again, perhaps they have one in mind and I just haven't read it. [1] https://aclanthology.org/2020.emnlp-main.170/ https://aclanthology.org/2020.emnlp-main.170/
  • vessenes 2y ago
    I believe he’s talking about some sort of ‘energy as measured by distance from the models understanding of the world’ as in quite literally a world model. But again I’m ignorant, hence the post!
    • tyronehed 2y ago
      When an architecture is based around world model building, then it is a casual outcome that similar concepts and things end up being stored in similar places. They overlap. As soon as your solution starts to get mathematically complex, you are departing from what the human brain does. Not saying that in some universe it might be possible to make a statistical intelligence, but when you go that direction you are straying away from the only existing intelligences that we know about. The human brain. So the best solutions will closely echo neuroscience.
    • deepsquirrelnet 2y ago
      In some respects that sounds similar to what we already do with reward models. I think with GRPO, the “bag of rewards” approach doesn’t strike me as terribly different. The challenge is in building out a sufficient “world” of rewards to adequately represent more meaningful feedback-based learning. While it sounds nice to reframe it like a physics problem, it seems like a fundamentally flawed idea, akin to saying “there is a closed form solution to the question of how should I live.” The problem isn’t hallucinations, the problem is that language and relativism are inextricably linked.
• tyronehed 2y ago
  Any transformer based LLM will never achieve AGI because it's only trying to pick the next word. You need a larger amount of planning to achieve AGI. Also, the characteristics of LLMs do not resemble any existing intelligence that we know of. Does a baby require 2 years of statistical analysis to become useful? No. Transformer architectures are parlor tricks. They are glorified Google but they're not doing anything or planning. If you want that, then you have to base your architecture on the known examples of intelligence that we are aware of in the universe. And that's not a transformer. In fact, whatever AGI emerges will absolutely not contain a transformer.
  • flawn 2y ago
    It's not about just picking the next word here, that doesn't at all refuse whether Transformers can achieve AGI. Words are just one representation of information. And whether it resembles any intelligence we know is also not an argument because there is no reason to believe that all intelligence is based on anything we've seen (e.g us, or other animals). The underlying architecture of Attention & MLPs can surely still depict something which we could call an AGI, and in certain tasks it surely can be considered an AGI already. I also don't know for certain whether we will hit any roadblocks or architectural asymptotes but I haven't come across any well-founded argument that Transformers definitely could not reach AGI.
  • visarga 2y ago
    The transformer is a simple and general architecture. Being such a flexible model, it needs to learn "priors" from data, it makes few assumptions on its distribution from the start. The same architecture can predict protein folding and fluid dynamics. It's not specific to language. We on the other hand are shaped by billions of years of genetic evolution, and 200k years of cultural evolution. If you count the total number of words spoken by 110 billion people who ever lived, assuming 1B estimated words per human during their lifetime, it comes out to 10 million times the size of GPT-4's training set. So we spent 10 million more words discovering than it takes the transformer to catch up. GPT-4 used 10 thousand people's worth of language to catch up all that evolutionary finetuning.
    • 2y ago
• tyronehed 2y ago
  The alternative architectures must learn from streaming data, must be error tolerant and must have the characteristic that similar objects or concepts much naturally come near to each other. They must naturally overlap.
• bitwize 2y ago
  Ever hear of Dissociated Press? If not, try the following demonstration. Fire up Emacs and open a text file containing a lot of human-readable text. Something off Project Gutenberg, say. Then say M-x dissociated-press and watch it spew hilarious, quasi-linguistic garbage into a buffer for as long as you like. Dissociated Press is a language model. A primitive, stone-knives-and-bearskins language model, but a language model nevertheless. When you feed it input text, it builds up a statistical model based on a Markov chain, assigning probabilities to each character that might occur next, given a few characters of input. If it sees 't' and 'h' as input, the most likely next character is probably going to be 'e', followed by maybe 'a', 'i', and 'o'. 'r' might find its way in there, but 'z' is right out. And so forth. It then uses that model to generate output text by picking characters at random given the past n input characters, resulting in a firehose of things that might be words or fragments of words, but don't make much sense overall. LLMs are doing the same thing. They're picking the next token (word or word fragment) given a certain number of previous tokens. And that's ALL they're doing. The only differences are really matters of scale: the tokens are larger than single characters, the model considers many, many more tokens of input, and the model is a huge deep-learning model with oodles more parameters than a simple Markov chain. So while Dissociated Press churns out obvious nonsensical slop, ChatGPT churns out much, much more plausible sounding nonsensical slop. But it's still just rolling them dice over and over and choosing from among the top candidates of "most plausible sounding next token" according to its actuarial tables. It doesn't think. Any thinking it appears to do has been pre-done by humans, whose thoughts are then harvested off the internet and used to perform macrodata refinement on the statistical model. Accordingly, if you ask ChatGPT a question, it may well be right a lot of the time. But when it's wrong, it doesn't know it's wrong, and it doesn't know what to do to make things right. Because it's just reaching into a bag of refrigerator magnet poetry tiles, weighted by probability of sounding good given the current context, and slapping whatever it finds onto the refrigerator. Over and over. What I think Yann LeCun means by "energy" above is "implausibility". That is, the LLM would instead grab a fistful of tiles -- enough to form many different responses -- and from those start with a single response and then through gradient descent or something optimize it to minimize some statistical "bullshit function" for the entire response, rather than just choosing one of the most plausible single tiles each go. Even that may not fix the hallucination issue, but it may produce results with fewer obvious howlers.
• janalsncm 2y ago
  I am an MLE not an expert. However, it is a fundamental problem that our current paradigm of training larger and larger LLMs cannot ever scale to the precision people require for many tasks. Even in the highly constrained realm of chess, an enormous neural net will be outclassed by a small program that can run on your phone. https://arxiv.org/pdf/2402.04494 https://arxiv.org/pdf/2402.04494
  • throw310822 2y ago
    > Even in the highly constrained realm of chess, an enormous neural net will be outclassed by a small program that can run on your phone. This is true also for the much bigger neural net that works in your brain, and even if you're the world champion of chess. Clearly your argument doesn't hold water.
    • janalsncm 2y ago
      For the sake of argument let’s say an artificial neural net is approximately the same as the brain. It sounds like you agree with me that smaller programs are both more efficient and more effective than a larger neural net. So you should also agree with me that those who say the only path to AGI is LLM maximalism are misguided.
      • jpadkins 2y ago
        smaller programs are better than artificial or organic neural net for constrained problems like chess. But chess programs don't generalize to any other intelligence applications, like how organic neural nets do today.
      • throw310822 2y ago
        > It sounds like you agree with me that smaller programs are both more efficient and more effective than a larger neural net. At playing chess. (But also at doing sums and multiplications, yay!) > So you should also agree with me that those who say the only path to AGI is LLM maximalism are misguided. No. First of all, it's a claim you just made up. What we're talking about is people saying that LLMs are not the path to AGI- an entirely different claim. Second, assuming there's any coherence to your argument, the fact that a small program can outclass an enormous NN is irrelevant to the question of whether the enormous NN is the right way to achieve AGI: we are "general intelligences" and we are defeated by the same chess program. Unless you mean that achieving the intelligence of the greatest geniuses that ever lived is still not enough.
• deleted 2y ago
  [deleted]
• jurschreuder 2y ago
  This concept comes from Hopfield networks. If two nodes are on, but the connection between them is negative, this causes energy to be higher. If one of those nodes switches off, energy is reduced. With two nodes this is trivial. With 10 nodes it's more difficult to solve, and with billions of nodes it is impossible to "solve". All you can do then is try to get the energy as low as possible. This way also neural networks can find out "new" information, that they have not learned, but is consistent with the constraints they have learned about the world so far.
  • vessenes 2y ago
    So, what’s modeled as a “node” in an EBM, and what’s modeled as a connection? Are they vectors in a tensor, (well I suppose almost certainly that’s a yes). Do they run side by side a model that’s being trained? Is the node connectivity architecture fixed or learned?
• deleted 2y ago
  [deleted]
• d--b 2y ago
  Well, it could be argued that the “optimal response” ie the one that sorta minimizes that “energy” is sorted by LLMs on the first iteration. And further iterations aren’t adding any useful information and in fact are countless occasions to veer off the optimal response. For example if a prompt is: “what is the Statue of Liberty”, the LLMs first output token is going to be “the”, but it kinda already “knows” that the next ones are going to be “statue of liberty”. So to me LLMs already “choose” a response path from the first token. Conversely, a LLM that would try and find a minimum energy for the whole response wouldn’t necessarily stop hallucinating. There is nothing in the training of a model that says that “I don’t know” has a lower “energy” than a wrong answer…
• rglover 2y ago
  Not an ML researcher, but implementing these systems has shown this opinion to be correct. The non-determinism of LLMs is a feature, not a bug that can be fixed. As a result, you'll never be able to get 100% consistent outputs or behavior (like you hypothetically can with a traditional algorithm/business logic). And that has proven out in usage across every model I've worked with. There's also an upper-bound problem in terms of context where every LLM hits some arbitrary amount of context that causes it to "lose focus" and develop a sort of LLM ADD. This is when hallucinations and random, unrequested changes get made and a previously productive chat spirals to the point where you have to start over.
  • numba888 2y ago
    > where every LLM hits some arbitrary amount of context that causes it to "lose focus" and develop a sort of LLM ADD. Humans brains have the same problem. As any intelligence probably. Solution for this is structural thinking. One piece at a time, often top-down. Educated humans do it, LLM can be orchestrated to do it too. Effective context window will be limited even though some claim millions of tokens.
• EEgads 2y ago
  Yann LeCun understands this is an electrical engineering and physical statistics of machine problem and not a code problem. The physics of human consciousness are not implemented in a leaky symbolic abstraction but the raw physics of existence. The sort of autonomous system we imagine when thinking AGI must be built directly into substrate and exhibit autonomous behavior out of the box. Our computers are blackboxes made in a lab without centuries of evolving in the analog world, finding a balance to build on. They either can do a task or cannot. Obviously from just looking at one we know how few real world tasks it can just get up and do. Code isn’t magic, it’s instruction to create a machine state. There’s no inherent intelligence to our symbolic logic. It’s an artifact of intelligence. It cannot imbue intelligence into a machine.
• bobosha 2y ago
  I argue that JEPA and its Energy-Based Model (EBM) framework fail to capture the deeply intertwined nature of learning and prediction in the human brain—the “yin and yang” of intelligence. Contemporary machine learning approaches remain heavily reliant on resource-intensive, front-loaded training phases. I advocate for a paradigm shift toward seamlessly integrating training and prediction, aligning with the principles of online learning. Disclosure: I am the author of this paper. Reference: (PDF) Hydra: Enhancing Machine Learning with a Multi-head Predictions Architecture. Available from: https://www.researchgate.net/publication/381009719_Hydra_Enhancing_Machine_Learning_with_a_Multi-head_Predictions_Architecture https://www.researchgate.net/publication/381009719_Hydra_Enh... [accessed Mar 14, 2025].
  • vessenes 2y ago
    Thank you. So, quick q - it would make sense to me that JEPA is an outcome of the YLC work; would you say that’s the case?
  • esafak 2y ago
    So you believe humans spend more energy on prediction, relative to computers? Isn't that because personal computers are not powerful enough to train big models, and most people have no desire to? It is more economically efficient to socialize the cost of training, as is done. Are you thinking of a distributed training, where we split the work and cost? That could happen when robots become more widespread.
    • bobosha 2y ago
      The human brain operates at just 25W of power—less than the monitor you're likely using right now—whereas AI models like ChatGPT consume nearly 1GWh every 24 hours! As I discuss in the paper, predictive coding suggests that the brain actively generates predictions and compares them to incoming sensory data (vision, hearing, etc.), prioritizing anomalies. Its efficiency stems from a hierarchical memory system that continuously updates only the "deltas"—the differences that matter. Embracing this approach could lead to a paradigm shift, enabling the development of significantly more energy-efficient AI in the future.
• inimino 2y ago
  I have a paper coming up that I modestly hope will clarify some of this. The short answer should be that it's obvious LLM training and inference are both ridiculously inefficient and biologically implausible, and therefore there has to be some big optimization wins still on the table.
  • jedberg 2y ago
    > and biologically implausible I really like this approach. Showing that we must be doing it wrong because our brains are more efficient and we aren't doing it like our brains. Is this a common thing in ML papers or something you came up with?
    • esafak 2y ago
      Evolution does not need to converge on the optimum solution. Have you heard of https://en.wikipedia.org/wiki/Bio-inspired_computing https://en.wikipedia.org/wiki/Bio-inspired_computing ?
      • jedberg 2y ago
        It does not, you're right. But it's an interesting way to approach the problem never the less. And given that we definitely aren't as efficient as a human brain right now, it makes sense to look at the brain for inspiration.
      • parsimo2010 2y ago
        I don't think GP was implying that brains are the optimum solution. I think you can interpret GP's comments like this- if our brains are more efficient than LLMs, then clearly LLMs aren't optimally efficient. We have at least one data point showing that better efficiency is possible, even if we don't know what the optimal approach is.
        • esafak 2y ago
          I agree. Spiking neural networks are usually mentioned in this context, but there is no hardware ecosystem behind them that can compete with Nvidia and CUDA.
• snats 2y ago
  Not an insider but imo the work on diffusion language models like LLaDA is really exciting. It's pretty obvious that LLMs are good but they are pretty slow. And in a world where people want agents you want a lot of the time something that might not be that smart but is capable of going really fast + searches fast. You only need to solve search in a specific domain for most agents. You don't need to solve the entire knowledge of human history in a single set of weights
• eximius 2y ago
  I believe that so long as weights are fixed at inference time, we'll be at a dead end. Will Titans be sufficiently "neuroplastic" to escape that? Maybe, I'm not sure. Ultimately, I think an architecture around "looping" where the model outputs are both some form of "self update" and "optional actionality" such that interacting with the model is more "sampling from a thought space" will be required.
  • mft_ 2y ago
    Very much this. I’ve been wondering why I’ve not seen it much discussed.
    • jononor 2y ago
      There are many roadblocks to continual learning still. Most current models and training paradigms are very vulnerable to catastrophic forgetting. And are very sample inefficient. And we/the methods are not so good at separating what is "interesting" (should be learned) vs "not". But this is being researched, for example under the topic of open ended learning, active inference, etc.
      • chriskanan 2y ago
        As a leader in the field of continual learning, I somewhat agree, but I'd say that catastrophic forgetting is largely resolved. The problem is that the continual learning community largely has become insular and is mostly focusing on toy problems that don't matter, where they will even avoid good solutions for nonsensical reasons. For example, reactivation / replay / rehearsal works well for mitigating catastrophic forgetting almost entirely, but a lot of the continual learning community mostly dislikes it because it is very effective. A lot of the work is focusing on toy problems and they refuse to scale up. I wrote this paper with some of my colleagues on this issue, although with such a long author list it isn't as focused as I would have liked in terms of telling the continual learning community to get out of its rut such that they are writing papers that advance AI rather than are just written for other continual learning researchers: https://arxiv.org/abs/2311.11908 https://arxiv.org/abs/2311.11908 The majority are focusing on the wrong paradigms and the wrong questions, which blocks progress towards the kinds of continual learning needed to make progress towards creating models that think in latent space and enabling meta-cognition, which would then give architectures the ability to avoid hallucinations by knowing what they don't know.
• simne 2y ago
  I'm not deep researcher, more like amateur, but could explain some things. Most problem with current approach, to grow abilities, need to add more neurons, but this is not just energy consuming, but also knowledge consuming, mean, at GPT-4 level all text sources of humanity already exhausted and model become essentially overfitted. So looks like multi-modal models appear not because so good, but because they could learn on additional sources (audio/video). I seen few approaches to overcome problem of overfitting, but as I understand not exist universal solution. For example, tried approach to create from current texts some synthetic training data, but this idea is limited by definition. So, current LLMs appear to hit dead end, and researchers now trying to find exit from this dead end. I believe, nearest years somebody will invent some universal solution (probably, complex of approaches) or suggest another architecture, and progress of AI will continue.
• giantg2 2y ago
  I feel like some hallucinations aren't bad. Isn't that basically what a new idea is - a hallucination of what could be? The ability to come up with new things, even if they're sometimes wrong, can be useful and happen all the time with humans.
  • hn_user82179 2y ago
    That’s a really interesting thought. I think the key part (as a consumer of AI tools) would be identifying the things that are guesses vs deductions vs complete accurate based on the training data. I would happily look up or think about the output parts that are possibly hallucinated myself but we don’t currently get that kind of feedback. Whereas a human could list things out that they know, and then highlight the things they making educated guesses about, which makes it easier to build upon.
    • giantg2 2y ago
      To be fair most people don't give you that level of detail. But I agree
  • chriskanan 2y ago
    Hallucinations is a terrible term for this. We want models that can create new ideas and make up stories. The problem is that they will give false answers to questions with factual answers, and that they don't realize this. In humans, this is known as confabulation, and it happens due to various forms of brain damage, especially with damage to orbitofrontal cortex (part of prefrontal cortex). David Rumelhart, who was the main person who came up with backpropagation in a paper co-authored with Geoff Hinton, actually got Pick's disease which specifically results in damage to prefrontal cortex and people with that disease exhibit a lot of the same problems we have with today's LLMs:
• schainks 2y ago
  This seems really intuitive to me. If I can express something concisely and succinctly because I understand it, I will literally spend less energy to explain it.
• jiggawatts 2y ago
  My observation from the outside watching this all unfold is that not enough effort seems to be going into the training schedule. I say schedule because the “static data once through” is the root of the problem in my mind is one of the root problems. Think about what happens when you read something like a book. You’re not “just” reading it, you’re also comparing it to other books, other books by the same author, while critically considering the book recommendations made by your friend. Any events in the book get compared to your life experience, etc… LLM training does none of this! It’s a once-through text prediction training regime. What this means in practice is that an LLM can’t write a review of a book unless it has read many reviews already. They have, of course, but the problem doesn’t go away. Ask an AI to critique book reviews and it’ll run out of steam because it hasn’t seen many of those. Critiques of critiques is where they start falling flat on their face. This kind of meta-knowledge is precisely what experts accumulate. As a programmer I don’t just regurgitate code I’ve seen before with slight variations — instead I know that mainstream criticisms of micro services misses their key benefit of extreme team scalability! This is the crux of it: when humans read their training material they are generating an “n+1” level in their mind that they also learn. The current AI training setup trains the AI only the “n”th level. This can be solved by running the training in a loop for several iterations after base training. The challenge of course is to develop a meaningful loss function. IMHO the “thinking” model training is a step in the right direction but nowhere near enough to produce AGI all by itself.
• infamouscow 2y ago
  There was an article about in the March 2025 issue of Communications of the ACM: https://dl.acm.org/doi/pdf/10.1145/3707200 https://dl.acm.org/doi/pdf/10.1145/3707200
• danielmarkbruce 2y ago
  Whether or not he's right big picture, the specific thing about runaway tokens is dumb as hell.
• ilaksh 2y ago
  I don't think you need to be an ML researcher to understand his point of view. He wants to do fundamental research. Optimizing LLMs is not fundamental research. There are numerous other potential approaches, and it's obvious that LLMs have weaknesses that other approaches could tackle. If he was Hinton's age then maybe he would also want to retire and be happy with transformers and LLMs. He is still an ambitious researcher that wants to do foundational research to get to the next paradigm. Having said all of that, it is a misjudgement for him to be disparaging the incredible capabilities of LLMs to the degree he has.
  • moron4hire 2y ago
    > it is a misjudgement for him to be disparaging the incredible capabilities of LLMs to the degree he has. Jeez, you'd think he kicked your dog.
• jmpeax 2y ago
  A transformer will attend to previous tokens, and so it is free to ignore prior errors. I don't get LeCun's comment on error propagation, and look forward to a more thorough exposition of the problem.
• akomtu 2y ago
  The next-gen LLMs are going to use something like mipmaps in graphics: a stack of progressively smaller versions of the image, with a 1x1 image at the top. The same concept applies to text. When you're writing something, your have a high-level idea in mind that serves as a guide. That idea is such a mipmap. Perhaps the next-gen LLMs will be generating a few parallel sequencies, the top-level will be a slow-pace anchor and the bottom-level being the actual text that depends on slower upper levels.
• bashfulpup 2y ago
  He's right but at the same time wrong. Current AI methods are essentially scaled up methods that we learned decades ago. These long horizon (agi) problems have been there since the very beginning. We have never had a solution to them. RL assumes we know the future which is a poor proxy. These energy based methods fundamentally do very little that an RNN didn't do long ago. I worked on higher dimensionality methods which is a very different angle. My take is that it's about the way we scale dependencies between connections. The human brain makes and breaks a massive amount of nueron connections daily. Scaling the dimensionality would imply that a single connection could be scalled to encompass significantly more "thoughts" over time. Additionally the true to solution to these problems are likely to be solved by a kid with a laptop as much as an top researcher. You find the solution to CL on a small AI model (mnist) you solve it at all scales.
  • nradov 2y ago
    For a kid with a laptop to solve it would require the problem to be solvable with current standard hardware. There's no evidence for that. We might need a completely different hardware paradigm.
    • bashfulpup 2y ago
      Also possible and a fair point. My point is that it's a "tiny" solution that we can scale. I could revise that by saying a kid with a whiteboard. It's an einstein×10 moment so who know when that'll happen.
  • haolez 2y ago
    Not exactly related, but I wonder sometimes if the fact that the weights in current models are very expansive to change is a feature and not a "bug". Somehow, it feels harder to trust a model that could evolve over time. It's performance might even degrade. That's a steep price to pay for having memory built in and a (possibly) self-evolving model.
    • bashfulpup 2y ago
      We degrade, and I think we are far more valuable than one model.
• bravura 2y ago
  Okay I think I qualify. I'll bite. LeCun's argument is this: 1) You can't learn an accurate world model just from text. 2) Multimodal learning (vision, language, etc) and interaction with the environment is crucial for true learning. He and people like Hinton and Bengio have been saying for a while that there are tasks that mice can understand that an AI can't. And that even have mouse-level intelligence will be a breakthrough, but we cannot achieve that through language learning alone. A simple example from "How Large Are Lions? Inducing Distributions over Quantitative Attributes" (https://arxiv.org/abs/1906.01327 https://arxiv.org/abs/1906.01327) is this: Learning the size of objects using pure text analysis requires significant gymnastics, while vision demonstrates physical size more easily. To determine the size of a lion you'll need to read thousands of sentences about lions, or you could look at two or three pictures. LeCun isn't saying that LLMs aren't useful. He's just concerned with bigger problems, like AGI, which he believes cannot be solved purely through linguistic analysis. The energy minimization architecture is more about joint multimodal learning. (Energy minimization is a very old idea. LeCun has been on about it for a while and it's less controversial these days. Back when everyone tried to have a probabilistic interpretation of neural models, it was expensive to compute the normalization term / partition function. Energy minimization basically said: Set up a sensible loss and minimize it.)
  • cm2012 2y ago
    This seems strongly backed up by Claude Plays Pokemon
    • fewhil 2y ago
      Isn't Claude Plays Pokemon using image input in addition to text? Not that it's perfect at it (some of its most glaring mistakes are when it just doesn't seem to understand what's on the screen correctly).
      • cm2012 2y ago
        Yes but because it's trained on text and in the backend, images are converted to tokens, it is absolutely dogshit at navigation and basic puzzles. It can't figure out what Squirrels can about how to achieve goals in a maze.
• itkovian_ 2y ago
  The fundamental distinction is usually made to contrastive approaches (i.e. make correct more likely, make everything else we just compared unlikely). Ebms are "only what is correct is more likely and the default for everything is unlikely" This is obviously an extremely high level simplification, but that's the core of it.
  • itkovian_ 2y ago
    And in this categorization auto regressive llms are contrastive due to the cross entropy loss.
• adamnemecek 2y ago
  We are actually working on scaling energy-based models http://traceoid.ai http://traceoid.ai
  • nickpsecurity 2y ago
    I hope you succeed. I've downloaded and at least skimmed hundreds of papers ML, many alternative architectures. A subset of them built prototypes that claimed good results on benchmarks. Of those, many didn't pass further scrutiny due to various failures. Those that did pass often failed on real-world tasks despite doing well on benchmarks. That we're so jaded by failures of published models makes us even more skeptical of unpublished methods. So, each architectural advance needs published prototypes solving real-world problems. The smallest I've seen do useful stuff are in 100+M-3B range. There are also papers about testing advances with low, pretraining cost: BabyLM; GPT2 replications; MosaicBERT. Some do straight pre-training while others distill field-proven models. Alternative architectures would do well to crank out examples like this to prove themselves. Please, do build at least one of the above using your method. Post it to your site. Link to demos of the actual prototype in use. This might get an ecosystem going that builds on your ideas.
    • adamnemecek 2y ago
      How many of these were actually addressing scaling EBMs though? I'm guessing none.
      • nickpsecurity 2y ago
        Including yours. Your landing page has no architecture, model, or performance comparisons. It's non-existent. You need something more tangible for us to believe in. Remember that scientific method requires us to reject everything by default. Only after rigorous review of a working theory or prototype do we treat it as truth. Build what you want us to believe in. Let us see it smoke the competing models of similar size in key metrics. That will do more for you than anything else. Again, I hope you're right and I get to see energy-based models being highly competitive. I haven't.
• hansvm 2y ago
  One small point: Token selection at each step is fine (and required if you want to be able to additively/linearly/independently handle losses). The problem here is the high inaccuracies in each token (or, rather, their distributions). If you use more time and space to generate the token then those errors go down. If using more time and space cannot suffice then, by construction, energy minimization models and any other solution you can think of also can't reduce the errors far enough.
• deleted 2y ago
  [deleted]
• gian995 2y ago
  [dead]
• deleted 2y ago
  [deleted]
• nsonha 2y ago
  cane someone elaborate on the energy bit? I vaguely recall something similar in ML 101 way back in university. Was that not widely used?
• fovc 2y ago
  I wonder if the error propagation problem could be solved with a “branching” generator? Basically at every token you fork off N new streams, with some tree pruning policy to avoid exponential blowup. With a bit of bookkeeping you could make an attention mask to support the parallel streams in the same context sharing prefixes. Perhaps that would allow more of an e2e error minimization than the greedy generation algorithm in use today?
• nonagono 2y ago
  Many of his arguments make “logical” sense, but one way to evaluate them is: would they have applied equally well 5 years ago? and would that have predicted LLMs will never write (average) poetry, or solve math, or answer common-sense questions about the physical world reasonably well? Probably. But turns out scale is all we needed. So yeah, maybe this is the exact point where scale stops working and we need to drastically change architectures. But maybe we just need to keep scaling.
• __lbracket__ 2y ago
  [dead]
• AlexCoventry 2y ago
  Not an insider, but: I don't know about you, but I certainly don't generate text autoregressively, token by token. Also, pretty sure I don't learn by global updates based on taking the derivative of some objective function of my behavior with respect to every parameter defining my brain. So there's good biological reason to think we can go beyond the capabilities of current architectures. I think probably an example of the kind of new architectures he supports is FB's Large Concept Models [1]. It's still a self-attention, autoregressive architecture, but the unit of regression is a sentence rather than a token. It maps sentences into a latent space via an autoencoder architecture, then has a transformer architecture in which the tokens are elements in that latent space. [1] https://arxiv.org/abs/2412.08821 https://arxiv.org/abs/2412.08821
• zmmmmm 2y ago
  So much of our fundamental scientific progress has been made by people who were considered crazy and their ideas delusional. Even mundane software engineering is done with layers of code review and automated tests because even the best engineers are still pretty bad at it. At a larger level, humanity itself seems to largely operate more like an ensemble method where many people in parallel solve problems and we empirically find who was "hallucinating". Which is just to say, it feels to me like there's a danger that the stochastic nature of outputs is fundamental to true creative intelligence and all attempts to stamp it out will result in lower accuracy overall. Rather we should be treating it more like we do actual humans and expect errors and put layers of process around things where it matters to make them safe.
• in3d 2y ago
  I don't think it's a coincidence that he is interested in non-LLM solutions, since he mentioned last year on Twitter that he doesn't have an internal monologue (I hope this is not taken as disparaging of him in any way). His criticisms of LLMs never made sense, and the success of reasoning models has shown him to be definitely wrong.
  • WiSaGaN 2y ago
    Yes, it is fascinating that humans can have such seemingly fundamental differences in how they function 'under the hood.' I also have a friend who is highly intelligent—they earned a STEM PhD from one of the best universities in the world—yet they struggle to follow complex movie plots, despite having a photographic memory. It would be interesting to develop mirror LLMs (or Large Anything Models) for all these different types of brains so we can study how exactly these traits manifest and interact.
• coderenegade 2y ago
  You could reframe the way LLMs are currently trained as energy minimization, since the Boltzmann distribution that links physics and information theory (and correspondingly, probability theory as well) is general enough to include all standard loss functions as special cases. It's also pretty straightforward to include RL in that category as well. I think what Lecun is probably getting at is that there's currently no way for a model to say "I don't know". Instead, it'll just do its best. For esoteric topics, this can result in hallucinations; for topics where you push just past the edge of well-known and easy-to-Google, you might get a vacuously correct response (i.e. repetition of correct but otherwise known or useless information). The models are trained to output a response that meets the criteria of quality as judged by a human, but there's no decent measure (that I'm aware of) of the accuracy of the knowledge content, or the model's own limitations. I actually think this is why programming and mathematical tasks have such a large impact on model performance: because they encode information about correctness directly into the task. So Yann is probably right, though I don't know that energy minimization is a special distinction that needs to be added. Any technique that we use for this task could almost certainly be framed as energy minimization of some energy function.
• iammrpayments 2y ago
  Why do they have to use the word “hallucination” when the model makes a mistake, if you tell your teacher or boss you didn’t get the answer wrong, you’ve hallucinated it, he will send you to the hospital.
• jongjong 2y ago
  It's weird that we don't have human-level AGI yet considering that we have AIs that are in some ways much smarter than humans. Top-end LLMs write better and faster than most humans. Top-end stable diffusion models can draw and render video much faster and with much more precision than the best human artists.
• prats226 2y ago
  I feel like success of LLM's have been combination of multiple factors coming together favourably: 1) Hardware becoming cheap enough to train models beyond a size where we could see emergent properties. Which is going to become cheaper and cheaper. 2) Model architecture which can in computationally less expensive manner being able to look at all inputs at the same time. CNN's, RNN's all succeded at smaller scale becuase they added inductive bias in architecture favourable to the input modality, but also became less generic. Attention is simpler in computation to scale it and also has lower inductive bias. 3) Unsupervised text on internet being source of data which requires light pre-processing hence almost no efforts wrt annotations etc reaching scale wrt scaling laws corrosponding to large size models. Also text data being diverse enough to be generic to encompass variety of topics, thoughts vs imagenet etc which is highly specific and costly to produce. Assuming that text only models will hit a bottleneck, then to have next generation models, in addition to a new architecture, we also have to find rich dataset which is even more generic and much richer in modalities and the architecture being able to natively ingest it? However something that is not predictible is how well the emergent properties can scale with model size further. Maybe few more unlocks like model being able to retain information well in spite of really large context length, ability to SFT on super complex reasoning tasks without disrupting weights enough to loose unsupervised learning might take us much further?
• trod1234 2y ago
  Not an ML researcher, but neither of those ideas are going to work. The token approach is inherently flawed because the tokens pre-suppose unique meaning when in fact they may not be unique. Said another way, it lacks properties that would be able to differentiate true from false because the differentiating input isn't included and cannot be derived from the inputs given. This goes to decidability.
• alok-g 2y ago
  >> believes LLMs will never fix hallucinations because, essentially, the token choice method at each step leads to runaway errors I may have an actual opinion on his viewpoint, however, I have a nitpick even before that. How exactly is 'LLM' defined here? Even if some energy-based thing is done, would some not call even that an LLM? If/when someone finds a way to fix it within the 'token choice' method, could some people not just start calling it something differently from 'LLM'. I think Yann needs to rephrase what exactly he wants to say.
• chriskanan 2y ago
  A lot of the responses seem to be answering a different question: "Why does LeCun think LLMs won't lead to AGI?" I could answer that, but the question you are asking is "Why does LeCun think hallucinations are inherent in LLMs?" To answer your question, think about how we train LLMs: We have them learn the statistical distribution of all written human language, such that given a chunk of text (a prompt, etc.) it then samples its output distribution to produces the next most likely token (word, sub-word, etc.) that should be produced and keeps doing that. It never learns how to judge what is true or false and during training it never needs to learn "Do I already know this?" It is just spoon fed information that it has to memorize and has no ability to acquire metacognition, which is something that it would need to be trained to attain. As humans, we know what we don't know (to an extent) and can identify when we already know something or don't already know something, such that we can say "I don't know." During training, an LLM is never taught to do this sort of introspection, so it never will know what it doesn't know. I have a bunch of ideas about how to address this with a new architecture and a lifelong learning training paradigm, but it has been hard to execute. I'm an AI professor, but really pushing the envelope in that direction requires I think a small team (10-20) of strong AI scientists and engineers working collaboratively and significant computational resources. It just can't be done efficiently in academia where we have PhD student trainees who all need to be first author and work largely in isolation. By the time AI PhD students get good, they graduate. I've been trying to find the time to focus on getting a start-up going focused on this. With Terry Sejnowski, I pitched my ideas to a group affiliated with Schmidt Sciences that funds science non-profits at around $20M per year for 5 years. They claimed to love my ideas, but didn't go for it....
  • emrah 2y ago
    Would you care to post your ideas somewhere online so others can read, critique, try etc?
    • random3 2y ago
      "we love your ideas" == no "when do you close the round?" = maybe money in the bank account = yes
• natch 2y ago
  Not formally an ML researcher but I’ve heard this (and similar from Melanie Mitchell) and it seems like ridiculous gatekeeping. There’s no real rule worthy of any respect imho that LLMs can’t be configured to get additional input data from images, audio, proprioception sensors, and any other modes. I can easily write a script to convert such data into tokens in any number of ways that would allow them to be fed in as tokens of a “language.” Convolutions for example. A real expert could do it even more easily or do a better job. And then don’t LeCun’s objections just evaporate? I don’t see why he thinks he has some profound point. For gods sake our own senses are heavily attenuated and mediated and it’s not like we actually experience raw reality ourselves, ever; we just feel like we do. LLMs can be extended to be situated. So much can be done. It’s like he’s seeing http in 1993 and saying it won’t be enough for the full web… well duh, but it’s a great start. Now go build on it. If anything the flaw in LLMs is how they maintain only one primary thread of prediction. But this is changing; having a bunch of threads working on the same problem and checking each other from different angles of the problem will be an obvious fix for a lot of issues.
• ic_fly2 2y ago
  I can recommend his introduction to energy models. It is a bit older but explains the idea very well.
• nilifah316 2y ago
  [dead]