6 ms·

What's actually happening in a LLM is many orders of magnitude more complex than a Markov chain. However, I agree that they're an amazing pedagogical tool for t

by lukev 3y ago

What's actually happening in a LLM is many orders of magnitude more complex than a Markov chain. However, I agree that they're an amazing pedagogical tool for the basic principles of how a LLM works, even to a non-technical audience. Many people try to "explain" LLMs starting with the principles of neural networks. This rarely works well: there are some significant conceptual leaps required. However, explaining that a LLMs are really just iterated next-word prediction based on a statistical model of the preceding words is something that most people can grok, and in a useful way: in my experience, it actually helps give people a useful intuition for why and how models hallucinate and what kind of things they're good/bad at. Markov chains are super simple iterated next-word prediction model, and can be explained in 15 minutes. It's a great way to explain LLMs to laypeople.

genman 3y ago
LLMs are Markov chains on neural network steroids.
wongarsu 3y ago
They are an especially useful tool right now, that might become less valuable as we get better at building LLMs. In principle the inner working of an LLM can be anything from a Markov-chain-like predictor to a beyond-human intelligence. Token prediction is the input/output format we chose, but you could communicate with a human in the same format and the human would show human-level intelligence. What makes Markov chains such a great pedagogic tool right now is that they share (approximately) the same interface, being a token predictor, and that current LLMs are much closer to the capabilities of a fantastically good Markov chain than those of an above-human intelligence.
- lukev 3y ago
  > In principle the inner working of an LLM can be anything from a Markov-chain-like predictor to a beyond-human intelligence. I'm afraid I have to disagree. Next-token prediction isn't just the interface we use for LLMs, it is fundamentally what they are, to the very core. The training and loss function of the foundation models are completely oriented towards next-token accuracy. Reasonable people can disagree about emergent behavior and if/how much the model is "planning ahead" in its weights (and what that could even mean) but it is emphatically not the case that the "next token" model is "just an interface". The analogy to human thought isn't accurate at all: we have our own recursive/iterative thought process, short and long term memory, decision making loops, etc. A LLMs has no "thought" outside of next-token prediction and no working memory aside from its context window. We don't fully understand all the emergent behavior of a transformer model but we definitely understand exactly what's happening at the mechanical level: each token is determined, one at a time, by solving an extremely complex but deterministic equation in which the model weights are coefficients, and the output of which is a probability distribution over the next token. There's no hidden intelligence or man behind the curtain. Whatever a LLM can do, next token prediction is how it does it.
  - famouswaffles 3y ago
    >The training and loss function of the foundation models are completely oriented towards next-token accuracy. This doesn't mean anything. Loss function and training only concern themselves with the result of the prediction. The in-between, the computation, training does not care except as a means to an end. It's not Input A > Output B. It's Input A > Computation > Output B. That Computation could quite literally be anything. And no, we do not automatically know what this computation might represent or if it even represents anything that would be understandable by us. If you train a Transformer meticulously on predicting the token that is the result of an addition, you might hope the computation it learns is some algorithm for addition but you wouldn't actually know until you attempted to and successfully probed the model.
    - lukev 3y ago
      But in an LLM it is not an arbitrary computation. Very specifically, it is a single forward pass through a neural network. Neural networks are very general function approximators so yes, there is some room for emergent behavior. But it could _not_ be "quite literally anything." It's plugging in values for a single (very big) equation. I think we do ourselves a disservice by pretending it's more of a black box than it is.
      - famouswaffles 3y ago
        How many passes it is irrelevant. You can perform any computation you like in a single pass if you have enough compute time. Trained transformers have limited computer time per token so each query is compute limited, but this is trivially increased, by increasing tokens, or by increasing dimensions in the next training round so that each token permits more compute time. A forward pass is not one big equation and I have no clue who you think it is. It's a series of computations, computations that depends on the query awaiting prediction. It's not even the same series of computations for each query because not all neurons are getting activated period and even when the same neurons get activated, they are not necessarily getting activated in the same way.
jjtheblunt 3y ago
Genuine question: what do you mean by many orders of magnitude more complex?
- thfuran 3y ago
  A typical demonstration markov chain probably has a length of around 3. A typical recent LLM probably has more than three billion parameters. That's not precisely apppes to apples, but the LLM is certainly vastly more complicated.
  - skrebbel 3y ago
    The way you describe it, it doesn't seem much more complicated to me, from a “how does it work” perspective, just way bigger.
    - lukev 3y ago
      The overall structure is the same as in "use statistics to predict the next token." With a Markov chain, the statistics are as simple as a mapping of n-grams to the number of times it appears in the corpus. With a LLM, the statistics are the result of 50 years of research in neural network architectures, terabytes of training data, and many millions of dollars worth of hardware, along with the teams to build and manage all the data pipelines. So yes, much more complicated.
      - tarsinge 3y ago
        You can have very complex calculations and a simple output. The complexity of the process to find the weights is not necessarily the same complexity than the process using them. The numbers don't get suddenly special because of how they were calculated (like 42 is just the number 42 even after 7 million years of calculations).
    - thfuran 3y ago
      If the minimal representation of a model of the behavior is "way bigger", why are you disputing that it's more complicated? What's the difference?
SilasX 3y ago
>However, explaining that a LLMs are really just iterated next-word prediction based on a statistical model of the preceding words is something that most people can grok, and in a useful way: in my experience, it actually helps give people a useful intuition for why and how models hallucinate and what kind of things they're good/bad at. At risk of showing my deficient understanding: that isn't actually true, is it?[1] LLMs do much more than simply predict subsequent text, AFAICT. I think back to the earlier LLM results that wowed the world with stuff like "now I have a model where you can ask it 'king - man + woman' ... and it returns 'queen'. Trippy!" That is pretty clearly not mere text prediction, at least not identically. Even nearly all of the computational "hard work" comes from the math of predicting subsequent letters, that whole computation is introducing a new primitive, the concept of combining, or doing math on, models to produce other models, and thereby making a statement about what words would come next in a variety of scenarios. That primitive is not present in the Markov predictor example, at least not without positing how a Markov predictor would be similarly transformed -- which, being very ignorant on the matter, I'm not sure is possible or not, but either way, leaves out a critical construct that enables ChatGPT to e.g. find limericks that aren't preceded by the command "Write me a limerick", as in my earlier comment[1]. [1] Earlier comment on why I think it's dubious to call LLM-based products such as ChatGPT "mere LMs": https://news.ycombinator.com/item?id=35472089 https://news.ycombinator.com/item?id=35472089
- sporkland 3y ago
  I'm pretty sure that is a property of the word2vec embeddings. I'm not 100% sure if the embeddings/hidden states in LLM's have the same property but my guess would be they do.
- pyinstallwoes 3y ago
  What you described was the vector space utility and mathematics which is the result of encoding it into a dimensional space (latent?). SOTA LLM use vector space dimensional embeddings in order to utilize a coordinate based intelligence “for free” with a vector space prediction mechanism on the index of context.
  - 3y ago
nextos 3y ago
HMMs are much more similar to S3, S4, and other deep state-space models than to transformers. In fact, HMMs are discrete shallow state-space models. I believe S4 and successors might become serious contenders to transformers. For a detailed tutorial, see https://srush.github.io/annotated-s4 https://srush.github.io/annotated-s4

by lukev 3y ago

genman 3y ago

LLMs are Markov chains on neural network steroids.

wongarsu 3y ago

They are an especially useful tool right now, that might become less valuable as we get better at building LLMs. In principle the inner working of an LLM can be anything from a Markov-chain-like predictor to a beyond-human intelligence. Token prediction is the input/output format we chose, but you could communicate with a human in the same format and the human would show human-level intelligence. What makes Markov chains such a great pedagogic tool right now is that they share (approximately) the same interface, being a token predictor, and that current LLMs are much closer to the capabilities of a fantastically good Markov chain than those of an above-human intelligence.

lukev 3y ago
> In principle the inner working of an LLM can be anything from a Markov-chain-like predictor to a beyond-human intelligence. I'm afraid I have to disagree. Next-token prediction isn't just the interface we use for LLMs, it is fundamentally what they are, to the very core. The training and loss function of the foundation models are completely oriented towards next-token accuracy. Reasonable people can disagree about emergent behavior and if/how much the model is "planning ahead" in its weights (and what that could even mean) but it is emphatically not the case that the "next token" model is "just an interface". The analogy to human thought isn't accurate at all: we have our own recursive/iterative thought process, short and long term memory, decision making loops, etc. A LLMs has no "thought" outside of next-token prediction and no working memory aside from its context window. We don't fully understand all the emergent behavior of a transformer model but we definitely understand exactly what's happening at the mechanical level: each token is determined, one at a time, by solving an extremely complex but deterministic equation in which the model weights are coefficients, and the output of which is a probability distribution over the next token. There's no hidden intelligence or man behind the curtain. Whatever a LLM can do, next token prediction is how it does it.
- famouswaffles 3y ago
  >The training and loss function of the foundation models are completely oriented towards next-token accuracy. This doesn't mean anything. Loss function and training only concern themselves with the result of the prediction. The in-between, the computation, training does not care except as a means to an end. It's not Input A > Output B. It's Input A > Computation > Output B. That Computation could quite literally be anything. And no, we do not automatically know what this computation might represent or if it even represents anything that would be understandable by us. If you train a Transformer meticulously on predicting the token that is the result of an addition, you might hope the computation it learns is some algorithm for addition but you wouldn't actually know until you attempted to and successfully probed the model.
  - lukev 3y ago
    But in an LLM it is not an arbitrary computation. Very specifically, it is a single forward pass through a neural network. Neural networks are very general function approximators so yes, there is some room for emergent behavior. But it could _not_ be "quite literally anything." It's plugging in values for a single (very big) equation. I think we do ourselves a disservice by pretending it's more of a black box than it is.
    - famouswaffles 3y ago
      How many passes it is irrelevant. You can perform any computation you like in a single pass if you have enough compute time. Trained transformers have limited computer time per token so each query is compute limited, but this is trivially increased, by increasing tokens, or by increasing dimensions in the next training round so that each token permits more compute time. A forward pass is not one big equation and I have no clue who you think it is. It's a series of computations, computations that depends on the query awaiting prediction. It's not even the same series of computations for each query because not all neurons are getting activated period and even when the same neurons get activated, they are not necessarily getting activated in the same way.

jjtheblunt 3y ago

Genuine question: what do you mean by many orders of magnitude more complex?

thfuran 3y ago
A typical demonstration markov chain probably has a length of around 3. A typical recent LLM probably has more than three billion parameters. That's not precisely apppes to apples, but the LLM is certainly vastly more complicated.
- skrebbel 3y ago
  The way you describe it, it doesn't seem much more complicated to me, from a “how does it work” perspective, just way bigger.
  - lukev 3y ago
    The overall structure is the same as in "use statistics to predict the next token." With a Markov chain, the statistics are as simple as a mapping of n-grams to the number of times it appears in the corpus. With a LLM, the statistics are the result of 50 years of research in neural network architectures, terabytes of training data, and many millions of dollars worth of hardware, along with the teams to build and manage all the data pipelines. So yes, much more complicated.
    - tarsinge 3y ago
      You can have very complex calculations and a simple output. The complexity of the process to find the weights is not necessarily the same complexity than the process using them. The numbers don't get suddenly special because of how they were calculated (like 42 is just the number 42 even after 7 million years of calculations).
  - thfuran 3y ago
    If the minimal representation of a model of the behavior is "way bigger", why are you disputing that it's more complicated? What's the difference?

SilasX 3y ago

>However, explaining that a LLMs are really just iterated next-word prediction based on a statistical model of the preceding words is something that most people can grok, and in a useful way: in my experience, it actually helps give people a useful intuition for why and how models hallucinate and what kind of things they're good/bad at. At risk of showing my deficient understanding: that isn't actually true, is it?[1] LLMs do much more than simply predict subsequent text, AFAICT. I think back to the earlier LLM results that wowed the world with stuff like "now I have a model where you can ask it 'king - man + woman' ... and it returns 'queen'. Trippy!" That is pretty clearly not mere text prediction, at least not identically. Even nearly all of the computational "hard work" comes from the math of predicting subsequent letters, that whole computation is introducing a new primitive, the concept of combining, or doing math on, models to produce other models, and thereby making a statement about what words would come next in a variety of scenarios. That primitive is not present in the Markov predictor example, at least not without positing how a Markov predictor would be similarly transformed -- which, being very ignorant on the matter, I'm not sure is possible or not, but either way, leaves out a critical construct that enables ChatGPT to e.g. find limericks that aren't preceded by the command "Write me a limerick", as in my earlier comment[1]. [1] Earlier comment on why I think it's dubious to call LLM-based products such as ChatGPT "mere LMs": https://news.ycombinator.com/item?id=35472089 https://news.ycombinator.com/item?id=35472089

sporkland 3y ago
I'm pretty sure that is a property of the word2vec embeddings. I'm not 100% sure if the embeddings/hidden states in LLM's have the same property but my guess would be they do.
pyinstallwoes 3y ago
What you described was the vector space utility and mathematics which is the result of encoding it into a dimensional space (latent?). SOTA LLM use vector space dimensional embeddings in order to utilize a coordinate based intelligence “for free” with a vector space prediction mechanism on the index of context.
- 3y ago

nextos 3y ago

HMMs are much more similar to S3, S4, and other deep state-space models than to transformers. In fact, HMMs are discrete shallow state-space models. I believe S4 and successors might become serious contenders to transformers. For a detailed tutorial, see https://srush.github.io/annotated-s4 https://srush.github.io/annotated-s4