9 ms·
The wall confronting large language models
- Scene_Cast2 1y agoThe paper is hard to read. There is no concrete worked-through example, the prose is over the top, and the equations don't really help. I can't make head or tail of this paper.
- joeldg 1y ago[dead]
- lumost 1y agoThis appears to be a position paper written by authors outside of their core field. The presentation of "the wall" is only through analogy to derivatives on the discrete values computer's operate in.
- joe_the_user 1y agoPaper seems to involve a series of analogies and equations. However, I think if the equations accepted, the "wall" is actually derived. The authors are computer scientists and people who work with large scale dynamic system. They aren't people who've actually produced an industry-scale LLM. However, I have to note that despite lots of practical progress in deep learning/transformers/etc systems, all the theory involved just analogies and equations of a similar sort, it's all alchemy and so people really good at producing these models seem to be using a bunch of effective rules of thumb and not any full or established models (despite books claiming to offer a mathematical foundation for enterprise, etc). Which is to say, "outside of core competence" doesn't mean as much as it would for medicine or something.
- ACCount37 1y agoNo, that's all the more reason to distrust major, unverified claims made by someone "outside of core competence". Applied demon summoning is ruled by empiricism and experimentation. The best summoners in the field are the ones who have a lot of practical experience and a sharp, honed intuition for the bizarre dynamics of the summoning process. And even those very summoners, specialists worth their weight in gold, are slaves to the experiment! Their novel ideas and methods and refinements still fail more often than they succeed! One of the first lessons you have to learn in the field is that of humility. That your "novel ideas" and "brilliant insights" are neither novel nor brilliant - and the only path to success lies through things small and testable, most of which do not survive the test. With that, can you trust the demon summoning knowledge of someone who has never drawn a summoning diagram?
- cwmoore 1y agoYour passions may have run away with you. https://news.ycombinator.com/item?id=45114753 https://news.ycombinator.com/item?id=45114753
- jibal 1y agoSomehow the game of telephone took us from "outside of their core field" (which wasn't true) to "outside of core competence" (which is grossly untrue). > One of the first lessons you have to learn in the field is that of humility. I suggest then that you make your statements less confidently.
- ForHackernews 1y agoThe freshly-summoned Gaap-5 was rumored to be the most accursed spirit ever witnessed by mankind, but so far it seems not dramatically more evil than previous demons, despite having been fed vastly more humans souls.
- lazide 1y agoPerhaps we’re reaching peak demon?
- lumost 1y agoI will venture my 2 cents, the equations kinda sorta look like something - but in no way approach a derivation of the wall. Specifically, I would have looked for a derivation which proved for one of/all of 1. Sequence Models relying on a markov chain, with and without summarization to extend beyond fixed length horizons. 2. All forms of attention mechanisms/dense layers. 3. A specific Transformer architecture. That there exists a limit on the representation or prediction powers of the model for tasks of all input/output token lengths or fixed size N input tokens/M output tokens. *Based On* a derived cost growth schedule for model size, data size, compute budgets. Separately, I would have expected a clear literature review of existing mathematical studies on LLM capabilities and limitations - for which there are *many*. Including studies that purport that Transformers can represent any program of finite pre-determined execution length.
- jibal 1y agoIf you look at their other papers, you will see that this is very much within their core field.
- lumost 1y agoTheir other papers are on simulation and applied chemistry. Where does their expertise in Machine Learning, or Large Language Models derive from? While it's not a requirement to have published in a field before publishing in a field. Having a coauthor who is from the target field or a peer review venue in that field as an entry point certainly raises credibility. From my limited claim to be in either Machine Learning or Large Language Models the paper does not appear to demonstrate what it claims. The author's language addresses the field of Machine Learning and LLM development as you would a young student - which does not help make their point.
- stonogo 1y agoIf you can't look at that publication list and see their expertise in macine learning, then it may be that they know more about your field than you know about theirs. Nothing wrong with that! Computational chemists use different terminology than computer scientists but there is significant overlap in the fields.
- JohnKemeny 1y agoHe's a chemist. Lots of chemists and physicists like to talk about computation without having any background in it. I'm not saying anything about the content, merely making a remark.
- chermi 1y agoYou're really not saying anything? Just a random remark with no bearing? Seth Lloyd, Wolpert, Landauer, Bennet, Fredkin, Feynman, Sejnowski, Hopfield, Zechinna, parisi,mezard, and zdebvora, Crutchfeld, Preskill, Deutsch, Manin, Szilard, MacKay.... I wish someone told them to shut up about computing. And I wouldn't dare claim von Neumann as merely a physicist, but that's where he was coming from. Oh and as much as I dislike him, Wolfram.
- deleted 1y ago[deleted]
- 18cmdick 1y agoGrifters in shambles.
- dcre 1y agoAlways fun to see a theoretical argument that something clearly already happening is impossible.
- ahartmetz 1y agoSo where are the recent improvements in LLMs proportional to the billions invested?
- dcre 1y agoValue for the money is not at issue in the paper!
- ahartmetz 1y agoI believe it is. They are saying that LLMs don't improve all that much from giving them more resources - and computing power (and input corpus size) is pretty proportional to money.
- BoredPositron 1y agoIt's not about value it's about the stagnation while throwing compute at the problem.
- dcre 1y agoExactly.
- crowbahr 1y agoReally? It sure seems like we're at the top of the S curve with LLMs. Wiring them up to talk the themselves as reasoning isn't scaling the core models, which have only made incremental gains for all the billions invested. There's plenty more room to grow with agents and tooling, but the core models are only slightly bumping YoY rather than the rocketship changes of 2022/23.
- 1y ago
- measurablefunc 1y agoThere is a formal extensional equivalence between Markov chains & LLMs but the only person who seems to be saying anything about this is Gary Marcus. He is constantly making the point that symbolic understanding can not be reduced to a probabilistic computation regardless of how large the graph gets it will still be missing basic stuff like backtracking (which is available in programming languages like Prolog). I think that Gary is right on basically all counts. Probabilistic generative models are fun but no amount of probabilistic sequence generation can be a substitute for logical reasoning.
- boznz 1y agological reasoning is also based on probability weights, most of the time that probability is so close to 100% that it can be assumed to be true without consequence.
- AaronAPU 1y agoStunningly, though I have been saying this for 20 years I’ve never come across someone else mention it until now.
- Certhas 1y agoI don't understand what point you're hinting at. Either way, I can get arbitrarily good approximations of arbitrary nonlinear differential/difference equations using only linear probabilistic evolution at the cost of a (much) larger state space. So if you can implement it in a brain or a computer, there is a sufficiently large probabilistic dynamic that can model it. More really is different. So I view all deductive ab-initio arguments about what LLMs can/can't do due to their architecture as fairly baseless. (Note that the "large" here is doing a lot of heavy lifting. You need _really_ large. See https://en.m.wikipedia.org/wiki/Transfer_operator https://en.m.wikipedia.org/wiki/Transfer_operator)
- klawed 1y ago> avoidance, which we also discuss in this paper, necessitates putting a much higher premium on insight and understanding of the structural characteristics of the problems being investigated. I wonder if the authors are aware of The Bitter Lesson
- CuriouslyC 1y agoThis article is accurate. That's why I'm investigating a bayesian symbolic lisp reasoner. It's incapable of hallucinating, it provides auditable traces which are actual programs and it kicks the crap out of LLMs at stuff like Arc-Agi, symbolic reasoning, logic programs, game playing, etc. I'm working on a paper where I show that the same model can break 80 on arc-agi, run the house by counting cards at blackjack, and solve complex mathematical word problems.
- leptons 1y agoLLMs are also incapable of "hallucinating", so maybe that isn't the buzzword you should be using.
- Animats 1y agoThat article is weird. They seem obsessed with nuclear reactors. Also, they misunderstand how floating point works. As one learns at high school, the continuous derivative is the limit of the discrete version as the displacement h is sent to zero. If our computers could afford infinite precision, this statement would be equally good in practice as it is in continuum mathematics. But no computer can afford infinite precision, in fact, the standard double-precision IEEE representation of floating numbers offers an accuracy around the 16th digit, meaning that numbers below 10−16 are basically treated as pure noise. This means that upon sending the displacement h below machine precision, the discrete derivatives start to diverge from the continuum value as roundoff errors then dominate the discretization errors. Yes, differentiating data has a noise problem. This is where gradient followers sometimes get stuck. A low pass filter can help by smoothing the data so the derivatives are less noisy. But is that relevant to LLMs? A big insight in machine learning optimization was that, in a high dimensional space, there's usually some dimension with a significant signal, which gets you out of local minima. Most machine learning is in high dimensional spaces but with low resolution data points.
- hatmanstack 1y agoHave no empirical feedback but subjectively it reads as though the authors are trying to proof their own intelligence through convolution and confusion. Pure AI slop IMHO.
- godelski 1y ago> A big insight in machine learning optimization was that I think the big insight was how useful this low order method still is. I think many people don't appreciate how new the study of high dimensional mathematics (let alone high dimensional statistics) actually is. I mean metric theory didn't really start till around the early 1900's. The big reason these systems are still mostly black boxes is because we still have a long way to go when it comes to understanding these spaces. But I think it is worth mentioning that low order approximations can still lock you out of different optima. While I agree the (Latent) Manifold Hypothesis pretty likely applies to many problems, this doesn't change the fact that even at relatively low dimensions (like 10D) are quite complex and have lots of properties that are unintuitive. With topics like language and images, I think it is safe to say that these still require operating in high dimensions. You're still going to have to contend with the complexities of the concentration of measure (an idea from the 70's). Still, I don't think anyone expected things to have worked out as well as they have. If anything I think it is more surprising we haven't run into issues earlier! I think there are still some pretty grand problems for AI/ML left. Personally this is why I push back against much of the hype. The hype machine is good if the end is in sight. But a hype machine creates a bubble. The gamble is if you call fill the bubble before it pops. But the risk is that if it pops before then, then it all comes crashing down. It's been a very hot summer but I'm worried that the hype will lead to a winter. I'd rather have had a longer summer than a hotter summer and a winter.
- phoenixhaber 1y ago[dead]
- pama 1y agoSauro, if you read this, please refrain from such low-content speculative statements: “On a loose but telling note, this is still three decades short of the number of neural connections in the human brain, 1015, and yet they consume some one hundred million times more power (GWatts as compared to the very modest 20 Watts required by our brains).” No human brain could have time to read all the materials of a modern LLM training run even if they lived and read eight hours a day since humans first appeared over 300,000 years ago. More to the point, inference of an LLM is way more energy efficient than human inference (see the energy costs of a B200 decoding a 671B parameter model and estimate the energy needed to write the equivalent of a human book worth of information as part of a larger batch). The main reason for the large energy costs of inference is that we are serving hundreds of millions of people with the same model. No humans have this type of scaling capability.
- vrighter 1y agoAnd yet, the human brain is still (way way wayyyyyyyyy) more capable than the LLMs at the actual thinking. They're as wide as an ocean and as shallow as a puddle in a pothole. And we didn't need to read all of the internet to do it. As for the "write a book" part, the LLM will write a book quickly sure, but a significant chunk of it will be bullshit. It will all be hallucinated, but the stopped clock will be right some of the time. No humans have this scaling capability? What do you call the reproductive cycle then? Lots of smaller brains, each one possible specialized in a few fields, together containing all of human knowledge. And you might say that's not the same thing!, to which I reply with "let's not kid ourselves, Mixture-of-Experts describes exactly this".
- throwaway314155 1y agoAgency may be better understood as Michael Levin's approach where e.g. a lifeform is something that can achieve the same goal using various methods (robust). Having said that, you can now simply move the goal posts to say that while one human cannot read that much in that amount of time - the collective of all humans certainly can - or at least they can approximate it in a similar fashion to LLM's. Since each of us can reap the benefits of the collective then the benefits are distributed back to the individuals as needed.
- awanderingmind 1y agoThere is a lot of focus in the comments on the authors' credentials and, apparently, their writing style. It is a pity, because I think their discussion of scaling is interesting, even if comparing LLMs to grid-based differential equation solvers might be unconventional (I haven't convinced myself whether the analogy is entirely apt/valid yet, but it could conceivably be).