5 ms·

On what evidence are you making this statement? We don't even know how existing LLMs work, not really, yet we're done? All signs are pointing to there still b

by mvkel 2y ago

On what evidence are you making this statement? We don't even know how existing LLMs work, not really, yet we're done? All signs are pointing to there still being no upper limit on results. Not yet. Just because it's not on the leaderboard yet, doesn't mean we've heard the bell.

bozey07 2y ago
Of course "we" know how LLMs work. Nobody would be able to make one, otherwise.
- mvkel 2y ago
  Just because you can make something doesn't mean you know why it's made. There are thousands of people around the world trying to reverse engineer what is going on in the billions or trillions of parameters in an LLM. It's a field called "Mechanistic Interpretability." The people who do the work jokingly call it "cursed" because it is so difficult and they have made so little progress so far. Literally nobody can predict before they are released what capabilities new models will have in them. And then, months after a model is released, people discover new abilities in it, such as decent chess playing. They are black boxes.
  - irthomasthomas 2y ago
    I predict that this is largely an illusion staged by the lack of publishing of the datasets and training regime used. Also an artefact of how evals have been done on a pass fail basis. So that an LLM that gets 90% of a question right is just as much a failure as one that gets 0% of the question. So that skills appear to emerge suddenly and surprisingly only due to the flawed way that we are forced to study them. Consider the training regime, and partial success towards a goal, and emergence is far less prevalent. There was a paper on that recently, I'll see if I can find.
    - mvkel 2y ago
      Until <5 years ago, AI was almost entirely a purely academic field, theoretical at that. Those same academics admit themselves that they're surprised at how well LLMs do considering how simple(?) rudimentary(?) the logic underneath is. I don't quite understand what you're saying. That these academics were being lazy by not properly investigating/publishing their findings? That doesn't seem right.
  - alsetmusic 2y ago
    They may be black boxes but that doesn't change that they are operating on statistics. I see no evidence that "AI" so far is anywhere near to cracking reasoning. It doesn't matter how magical their inner-workings. They have been trained to spit out plausible text and images (and more limited, video). All of that is imitation, nothing near thought.
- Gooblebrai 2y ago
  We were making and using electricity before we knew about the electron
- aprilthird2021 2y ago
  It's very commonly understood by those of us who actually produce and consume AI research that "knowing how" LLMs (and Neural Nets for that matter) work doesn't mean knowing how to build one. It means mathematically proving and understanding "how" the steps we put the LLM through when training are able to produce the output we get when testing. We know how to build it. We don't understand how it's producing the output it does based off what we give it
  - AstralStorm 2y ago
    It's a bit similar to the problem of neuroscience. We understand how a single neuron works pretty well, or even a small count of them. Even a few subsystems like balance or lower level vision. A bit on muscle control and endocrine system. We do not understand language, grammar, music, only partly emotions, or especially sentience and consciousness. Further, we don't understand how the disparate systems are integrated together.
    - mvkel 2y ago
      Great comparison. The comparison itself is pretty telling; that the brain and AI work so similarly in specific ways
  - kybernetikos 2y ago
    This still doesn't make sense to me. As far as I'm concerned the gold standard of understanding something is being able to construct a program that replicates it, which is exactly what we can do with LLMs. We know exactly how llms work (relatively simple maths), and to a large extent even why they work (backpropagation updates weights to more closely approximate the desired function). There are open questions relating to LLMs of course - we don't understand what the space of potential LLM-like things looks like and how the features in that space relate to subjective performance (although note that transformers were designed based on a theory that they would perform better, not just randomly generated or inspired by the muse). We also don't know to what extent the output of LLMs can be approximated by simpler symbolic systems, or how to extract such systems from LLMs when they do exist. Those are really interesting questions, but they're not questions about 'how LLMs work'. I dislike the 'LLMs are magic' framing that seems to be taking over the world. Nobody thinks that Taylor expansion is magical, but LLMs are doing the same sort of thing - approximating a function through a bunch of weights on a bunch of simpler functions. Just because the function we're approximating (intelligent output) is not known in advance (but can be sampled), and multi-dimensional does not fundamentally change how mysterious the process is.
visarga 2y ago
The upper limit is on human text, especially of very high quality and original. But if LLMs use other sources of learning signal, like AlphaZero who trained in pure self play, that has no upper limit. It's just a slow, iterative process, and it requires social aspects. The path ahead to AGI is deceitful, there are no gradients leading towards it. It's based on exploration and discovery. It works in populations for reasons of diversity - evolution is that way. And evolution is a slow process, not a singularity kind of event.
- binkethy 2y ago
  I find the presumption of an AGI to be an article of religious faith. I realized, at some point this past year, that silicon valley is a religious center emitting theology and doctrine. I am an apostate. Technology does not have any innate vector for linear development. The choice to amplify dystopian social trends remains a wrongful choice. So-called AI is bad for humanity and the planet, in finite terms at this moment in time. It's essence is wrongful. Turning bad ideas up to 11 does not make them good ideas.
lumost 2y ago
The bearish case would be that companies are now putting tens of billions into compute/data to beat GPT-4 - but aren't. However I think the initial ~6 month gap between ChatGPT and GPT-4 set unrealistic expectations on the people time required to turn out a new AI product.

by mvkel 2y ago

bozey07 2y ago

Of course "we" know how LLMs work. Nobody would be able to make one, otherwise.

mvkel 2y ago
Just because you can make something doesn't mean you know why it's made. There are thousands of people around the world trying to reverse engineer what is going on in the billions or trillions of parameters in an LLM. It's a field called "Mechanistic Interpretability." The people who do the work jokingly call it "cursed" because it is so difficult and they have made so little progress so far. Literally nobody can predict before they are released what capabilities new models will have in them. And then, months after a model is released, people discover new abilities in it, such as decent chess playing. They are black boxes.
- irthomasthomas 2y ago
  I predict that this is largely an illusion staged by the lack of publishing of the datasets and training regime used. Also an artefact of how evals have been done on a pass fail basis. So that an LLM that gets 90% of a question right is just as much a failure as one that gets 0% of the question. So that skills appear to emerge suddenly and surprisingly only due to the flawed way that we are forced to study them. Consider the training regime, and partial success towards a goal, and emergence is far less prevalent. There was a paper on that recently, I'll see if I can find.
  - mvkel 2y ago
    Until <5 years ago, AI was almost entirely a purely academic field, theoretical at that. Those same academics admit themselves that they're surprised at how well LLMs do considering how simple(?) rudimentary(?) the logic underneath is. I don't quite understand what you're saying. That these academics were being lazy by not properly investigating/publishing their findings? That doesn't seem right.
- alsetmusic 2y ago
  They may be black boxes but that doesn't change that they are operating on statistics. I see no evidence that "AI" so far is anywhere near to cracking reasoning. It doesn't matter how magical their inner-workings. They have been trained to spit out plausible text and images (and more limited, video). All of that is imitation, nothing near thought.
Gooblebrai 2y ago
We were making and using electricity before we knew about the electron
aprilthird2021 2y ago
It's very commonly understood by those of us who actually produce and consume AI research that "knowing how" LLMs (and Neural Nets for that matter) work doesn't mean knowing how to build one. It means mathematically proving and understanding "how" the steps we put the LLM through when training are able to produce the output we get when testing. We know how to build it. We don't understand how it's producing the output it does based off what we give it
- AstralStorm 2y ago
  It's a bit similar to the problem of neuroscience. We understand how a single neuron works pretty well, or even a small count of them. Even a few subsystems like balance or lower level vision. A bit on muscle control and endocrine system. We do not understand language, grammar, music, only partly emotions, or especially sentience and consciousness. Further, we don't understand how the disparate systems are integrated together.
  - mvkel 2y ago
    Great comparison. The comparison itself is pretty telling; that the brain and AI work so similarly in specific ways
- kybernetikos 2y ago
  This still doesn't make sense to me. As far as I'm concerned the gold standard of understanding something is being able to construct a program that replicates it, which is exactly what we can do with LLMs. We know exactly how llms work (relatively simple maths), and to a large extent even why they work (backpropagation updates weights to more closely approximate the desired function). There are open questions relating to LLMs of course - we don't understand what the space of potential LLM-like things looks like and how the features in that space relate to subjective performance (although note that transformers were designed based on a theory that they would perform better, not just randomly generated or inspired by the muse). We also don't know to what extent the output of LLMs can be approximated by simpler symbolic systems, or how to extract such systems from LLMs when they do exist. Those are really interesting questions, but they're not questions about 'how LLMs work'. I dislike the 'LLMs are magic' framing that seems to be taking over the world. Nobody thinks that Taylor expansion is magical, but LLMs are doing the same sort of thing - approximating a function through a bunch of weights on a bunch of simpler functions. Just because the function we're approximating (intelligent output) is not known in advance (but can be sampled), and multi-dimensional does not fundamentally change how mysterious the process is.

visarga 2y ago

The upper limit is on human text, especially of very high quality and original. But if LLMs use other sources of learning signal, like AlphaZero who trained in pure self play, that has no upper limit. It's just a slow, iterative process, and it requires social aspects. The path ahead to AGI is deceitful, there are no gradients leading towards it. It's based on exploration and discovery. It works in populations for reasons of diversity - evolution is that way. And evolution is a slow process, not a singularity kind of event.

binkethy 2y ago
I find the presumption of an AGI to be an article of religious faith. I realized, at some point this past year, that silicon valley is a religious center emitting theology and doctrine. I am an apostate. Technology does not have any innate vector for linear development. The choice to amplify dystopian social trends remains a wrongful choice. So-called AI is bad for humanity and the planet, in finite terms at this moment in time. It's essence is wrongful. Turning bad ideas up to 11 does not make them good ideas.

lumost 2y ago

The bearish case would be that companies are now putting tens of billions into compute/data to beat GPT-4 - but aren't. However I think the initial ~6 month gap between ChatGPT and GPT-4 set unrealistic expectations on the people time required to turn out a new AI product.