4 ms·

> Do any LLMs do this? If not, why can't they? Because they aren't knowledgeable. The marketing and at-first-blush impressions that LLMs leave as some kind of

> Do any LLMs do this? If not, why can't they? Because they aren't knowledgeable. The marketing and at-first-blush impressions that LLMs leave as some kind of actual being, no matter how limited, mask this fact and it's the most frustrating thing about trying to evaluate this tech as useful or not. To make an incredibly complex topic somewhat simple, LLMs train on a series of materials, in this case we'll talk words. It learns that "it turns out," "in the case of", "however, there is" are all words that naturally follow one another in writing, but it has no clue why one would choose one over the other beyond the other words which form the contexts in which those word series' appear. This process is repeated billions of times as it analyzes the structure of billions of written words until it arrives at a massive in scale statistical model of how likely it is that every word will be followed by every other word or punctuation mark. Having all that data available does mean an LLM can generate... words. Words that are pretty consistently spelled and arranged correctly in a way that reflects the language they belong to. And, thanks to the documents it trained on, it gains what you could, if you're feeling generous, call a "base of knowledge" on a variety of subjects, in that by the same statistical model, it has "learned" that "measure twice, cut once" is said often enough that it's likely good advice, but again, it doesn't know why that is, which would be: it optimizes your cuts and avoids wasting materials when building something to measure it, mark it, then measure it a second or even third time to make sure it was done correctly before you do the cut, which an operation that cannot be reversed. However that knowledge has a HARD limit in terms of what was understood within it's training data. For example, way back, a GPT model recommended using elmer's glue to keep pizza toppings attached when making a pizza. No sane person would suggest this, because glue... isn't food. But the LLM doesn't understand that, it takes the question: how do I keep toppings on pizza, and it says, well a ton of things I read said you should use glue to stick things together, and ships that answer out. This is why I firmly believe LLMs and true AI are just... not the same thing, at all, and I'm annoyed that we now call LLMs AI and AI AGI, because in my mind, LLMs do not demonstrate any intelligence at all.

smokel 2y ago
This explanation is only superficially correct, and there is more to it than simply predicting the next word. It is the way in which the prediction works, that leads to some form of intelligence.
ryoshu 2y ago
The glue on pizza thing was a bit more pernicious because of how the model came to that conclusion: SERPs. Google's LLM pulled the top result for that query from Reddit and didn't understand that the Reddit post was a joke. It took it as the most relevant thing and hilarity ensued. In that case the error was obvious, but these things become "dangerous" for that sort of use case when end users trust the "AI result" as the "truth".
- ryandrake 2y ago
  Treating "highest ranked," "most upvoted," "most popular," and "frequently cited" as a signal of quality or authoritativeness has proven to be a persistent problem for decades.
  - ithkuil 2y ago
    Depends on the metric. Humans who up-voted that material clearly thought it was worth. The problem is distinguishing the various reasons people think something is worth and using the right context. That requires a lot of intelligence. The fact that modern language models are able to model sentiment and sarcasm as well as they do is a remarkable achievement. Sure there is a lot of work to be done to improve that, especially at scale and in products where humans are expecting something more than a good statistical "success rate", but they actually expect the precision level they are used from professionally curated human sources.
  - cratermoon 2y ago
    In this case it was a loss of context. The original post was highly upvoted because in the context of jokes it was considered good. Take it out of that context and treat "most upvoted" as a signal that means something like authoritativeness and the result will be still be hilarious, but this time unintentionally so. Or in short, LLMs don't get satire.
skydhash 2y ago
LLMs are great machine learning tech. But what exactly are they learning? No ones knows, because we're just feeding it the internet (or a good part of it) and hoping something good comes out of the end. But so far, it just shows that it only learn the closeness of one unit (token, pixels block,...) to each other. But with no idea why they are close in the first place.
foobarian 2y ago
This sounds an awful lot like the old Markov chains we used to write for fun in school. Is the difference really just scale? There has got to be more to it.
- AlexCoventry 2y ago
  You can think of an autoregressive LLM as a Markov chain, sure. It's just sampling from a much more sophisticated distribution than the ones you wrote for fun did. That by itself is not much of an argument against LLMs, though.
- alabastervlog 2y ago
  They're Markov chain generators with weighting that looks many tokens back and assigns, based on a training corpus, higher weight ("attention") to tokens that are more likely to significantly influence the probability of later tokens ("evolutionary" might get greater weight than "the", for instance, though to be clear tokens aren't necessarily the same as words), then smears those various weights together before rolling its newly-weighted dice to come up with the next token. Throw in some noise-reduction that disregards too-low probabilities, and that's basically it. This dials down the usual chaos of Markov chains, and makes their output far more convincing. Yes, that's really what all this fuss is about. Very fancy Markov chains.
Terr_ 2y ago
> The marketing and at-first-blush impressions that LLMs leave as some kind of actual being, no matter how limited, mask this fact I like to highlight the fundamental difference between fictional qualities of a fictional character versus actual qualities of an author. I might make a program that generates a story about Santa Claus, but that doesn't mean Santa Claus is real or that I myself have a boundless capacity to care for all the children in the world. Many consumers are misled into thinking they are conversing with an "actual being", rather than contributing "then the user said" lines to a hidden theater script that has a helpful-computer character in it.

smokel 2y ago

This explanation is only superficially correct, and there is more to it than simply predicting the next word. It is the way in which the prediction works, that leads to some form of intelligence.

ryoshu 2y ago

The glue on pizza thing was a bit more pernicious because of how the model came to that conclusion: SERPs. Google's LLM pulled the top result for that query from Reddit and didn't understand that the Reddit post was a joke. It took it as the most relevant thing and hilarity ensued. In that case the error was obvious, but these things become "dangerous" for that sort of use case when end users trust the "AI result" as the "truth".

ryandrake 2y ago
Treating "highest ranked," "most upvoted," "most popular," and "frequently cited" as a signal of quality or authoritativeness has proven to be a persistent problem for decades.
- ithkuil 2y ago
  Depends on the metric. Humans who up-voted that material clearly thought it was worth. The problem is distinguishing the various reasons people think something is worth and using the right context. That requires a lot of intelligence. The fact that modern language models are able to model sentiment and sarcasm as well as they do is a remarkable achievement. Sure there is a lot of work to be done to improve that, especially at scale and in products where humans are expecting something more than a good statistical "success rate", but they actually expect the precision level they are used from professionally curated human sources.
- cratermoon 2y ago
  In this case it was a loss of context. The original post was highly upvoted because in the context of jokes it was considered good. Take it out of that context and treat "most upvoted" as a signal that means something like authoritativeness and the result will be still be hilarious, but this time unintentionally so. Or in short, LLMs don't get satire.

skydhash 2y ago

LLMs are great machine learning tech. But what exactly are they learning? No ones knows, because we're just feeding it the internet (or a good part of it) and hoping something good comes out of the end. But so far, it just shows that it only learn the closeness of one unit (token, pixels block,...) to each other. But with no idea why they are close in the first place.

foobarian 2y ago

This sounds an awful lot like the old Markov chains we used to write for fun in school. Is the difference really just scale? There has got to be more to it.

AlexCoventry 2y ago
You can think of an autoregressive LLM as a Markov chain, sure. It's just sampling from a much more sophisticated distribution than the ones you wrote for fun did. That by itself is not much of an argument against LLMs, though.
alabastervlog 2y ago
They're Markov chain generators with weighting that looks many tokens back and assigns, based on a training corpus, higher weight ("attention") to tokens that are more likely to significantly influence the probability of later tokens ("evolutionary" might get greater weight than "the", for instance, though to be clear tokens aren't necessarily the same as words), then smears those various weights together before rolling its newly-weighted dice to come up with the next token. Throw in some noise-reduction that disregards too-low probabilities, and that's basically it. This dials down the usual chaos of Markov chains, and makes their output far more convincing. Yes, that's really what all this fuss is about. Very fancy Markov chains.

Terr_ 2y ago

> The marketing and at-first-blush impressions that LLMs leave as some kind of actual being, no matter how limited, mask this fact I like to highlight the fundamental difference between fictional qualities of a fictional character versus actual qualities of an author. I might make a program that generates a story about Santa Claus, but that doesn't mean Santa Claus is real or that I myself have a boundless capacity to care for all the children in the world. Many consumers are misled into thinking they are conversing with an "actual being", rather than contributing "then the user said" lines to a hidden theater script that has a helpful-computer character in it.