3 ms·

I think an LLM can be used as a kind of lossy encyclopedia, but equating it directly to one isn't entirely accurate. The human mind is also, in a sense, a lossy

by kgeist 1y ago

I think an LLM can be used as a kind of lossy encyclopedia, but equating it directly to one isn't entirely accurate. The human mind is also, in a sense, a lossy encyclopedia. I prefer to think of LLMs as lossy predictors. If you think about it, natural "intelligence" itself can be understood as another type of predictor: you build a world model to anticipate what will happen next so you can plan your actions accordingly and survive. In the real world, with countless fuzzy factors, no predictor can ever be perfectly lossless. The only real difference, for me, is that LLMs are lossier predictors than human minds (for now). That's all there is to it. Whatever analogy you use, it comes down to the realization that there's always some lossiness involved, whether you frame it as an encyclopedia or not.

jbstack 1y ago
Are LLMs really lossier than humans? I think it depends on the context. Given any particular example, LLMs might hallucinate more and a human might do a better job at accuracy. But overall LLMs will remember far more things than a human. Ask a human to reproduce what they read in a book last year and there's a good chance you'll get either absolutely nothing or just a vague idea of what the book was about - in this context they can be up to 100% lossy. The difference here is that human memory decays over time while a LLM's memory is hardwired.
- sigmoid10 1y ago
  >Given any particular example, LLMs might hallucinate more and a human might do a better job at accuracy This drastically depends on the example. For average trivia questions, modern LLMs (even smaller, open ones) beat humans easily.
- ijk 1y ago
  I think what trips people up is that LLMs and humans are both lossy, but in different ways. The intuitions that we've developed around previous interactions are very misleading when applied to LLMs. When interacting with a human, we're used to being able to ask a question about topic X in context Y and assume that if you can answer it we can rely on you to be able to talk about it in the very similar context Z. But LLMs are bad at commutative facts; A=B and B=A can have different performance characteristics. Just because it can answer A=B does not mean it is good at answering B=A; you have to test them separately. I've seen researchers who should really know better screw this up, rendering their methodology useless for the claim they're trying to validate. Our intuition for how humans do things can be very misleading when working with LLMs.
- withinboredom 1y ago
  That's not exactly true. Every time you start a new conversation; you get a new LLM for all intents. Asking an LLM about an unrelated topic towards the end of a ~500 page conversation will get you vastly different results than at the beginning. If we could get to multi-thousand page contexts, it would probably be less accurate than a human, tbh.
  - jbstack 1y ago
    Yes, I should have clarified that I was referring to memory of training data, not of conversations.
    - withinboredom 1y ago
      Training data also deteriorates quite quickly as the context gets longer.
A_D_E_P_T 1y ago
> If you think about it, natural "intelligence" itself can be understood as another type of predictor: you build a world model to anticipate what will happen next so you can plan your actions accordingly and survive. Yes. Human intelligence consists of three things. First, groundedness: The ability to form a representation of the world and one’s place in it. Second, a temporal-spatial sense: A subjective and bounded idea of self in objective space and time. Third: A general predictive function which is capable of broad abstraction. At its most basic level, this third element enables man to acquire, process, store, represent, and continually re-acquire knowledge which is external to that man's subjective existence. This is calculation in the strictest sense. And it is the third element -- the strength, speed, and breadth of the predictive function -- which is synonymous with the word "intelligence." Higher animals have all three elements, but they're pretty hazy -- especially the third. And, in humans, short time horizons are synonymous with intellectual dullness. All of this is to say that if you have a "prediction machine" you're 90% of the way to a true "intelligence machine." It also, I think, suggests routes that might lead to more robust AI in the future. (Ground the AI, give it a limited physical presence in time and space, match its clocks to the outside world.)
- quonn 1y ago
  "Prediction" is hardly more than another term for inference. It's the very essence of machine learning. There is nothing new or useful in this concept.
  - A_D_E_P_T 1y ago
    Point is that it's also exactly analogous to human intelligence. There's almost nothing else to it.
    - devmor 1y ago
      This is how you spot hype nonsense - claims that anything is analogous to human intelligence. Even absent all other objections, we don't understand the human mind well enough to make a claim like that.
cubefox 1y ago
Another difference is that you are predicting future sensory experiences in real-time, while LLMs "predict" text which a "helpful, honest, harmless" assistant would produce.
somewhereoutth 1y ago
> you build a world model The foundational conceit (if you will) of LLMs is that they build a semantic (world) model to 'make sense' of their training. However it is much more likely that they are simply building a syntactic model in response to the training. As far as I know there is no evidence of a semantic model emerging.
- jebarker 1y ago
  Maybe I don’t have a precise enough definition of syntax and semantics, but it seems like it’s more than just syntactic since interchangeable tokens in the same syntax affect the semantics of the sentence. Or do you view completing a prompt such as “The president of the United States is?” as a syntax question?
- ijk 1y ago
  There's some evidence of valid relationships: you can build a map of Manhattan by asking about directions from each street corner and plotting the relations. This is still entirely referential, but in a way that a human would see some relation to the actual thing, albeit in a somewhat weird and alien way.
- deleted 1y ago
  [deleted]
- IanCal 1y ago
  Is this not addressed by othellogpt?
layer8 1y ago
Lossy is an incomplete characterization. LLMs are also much more fluctuating and fuzzy. You can get wildly varying output depending on prompting, for what should be the same (even if lossy) knowledge. There is not just loss during the training, but also loss and variation during inference. An LLM overall is a much less coherent and consistent thing than most humans, in terms of knowledge, mindset, and elucidations.
NoMoreNicksLeft 1y ago
Imagine having the world's most comprehensive encyclopedia at your literal fingertips, 24 hours a day, but being so lazy that you offload the hard work of thinking by letting retarded software pathologically lie to you and then blindly accepting the non-answers it spits at you rather than typing in two or three keywords to Wikipedia and skimming the top paragraph. >I prefer to think of LLMs as lossy predictors. I've started to call them the Great Filter. In the latest issue of the comic book Lex Luthor attempts to exterminate humanity by hacking the LLM and having it inform humanity that they can hold their breath underwater for 17 hours.

by kgeist 1y ago

jbstack 1y ago

Are LLMs really lossier than humans? I think it depends on the context. Given any particular example, LLMs might hallucinate more and a human might do a better job at accuracy. But overall LLMs will remember far more things than a human. Ask a human to reproduce what they read in a book last year and there's a good chance you'll get either absolutely nothing or just a vague idea of what the book was about - in this context they can be up to 100% lossy. The difference here is that human memory decays over time while a LLM's memory is hardwired.

sigmoid10 1y ago
>Given any particular example, LLMs might hallucinate more and a human might do a better job at accuracy This drastically depends on the example. For average trivia questions, modern LLMs (even smaller, open ones) beat humans easily.
ijk 1y ago
I think what trips people up is that LLMs and humans are both lossy, but in different ways. The intuitions that we've developed around previous interactions are very misleading when applied to LLMs. When interacting with a human, we're used to being able to ask a question about topic X in context Y and assume that if you can answer it we can rely on you to be able to talk about it in the very similar context Z. But LLMs are bad at commutative facts; A=B and B=A can have different performance characteristics. Just because it can answer A=B does not mean it is good at answering B=A; you have to test them separately. I've seen researchers who should really know better screw this up, rendering their methodology useless for the claim they're trying to validate. Our intuition for how humans do things can be very misleading when working with LLMs.
withinboredom 1y ago
That's not exactly true. Every time you start a new conversation; you get a new LLM for all intents. Asking an LLM about an unrelated topic towards the end of a ~500 page conversation will get you vastly different results than at the beginning. If we could get to multi-thousand page contexts, it would probably be less accurate than a human, tbh.
- jbstack 1y ago
  Yes, I should have clarified that I was referring to memory of training data, not of conversations.
  - withinboredom 1y ago
    Training data also deteriorates quite quickly as the context gets longer.

A_D_E_P_T 1y ago

> If you think about it, natural "intelligence" itself can be understood as another type of predictor: you build a world model to anticipate what will happen next so you can plan your actions accordingly and survive. Yes. Human intelligence consists of three things. First, groundedness: The ability to form a representation of the world and one’s place in it. Second, a temporal-spatial sense: A subjective and bounded idea of self in objective space and time. Third: A general predictive function which is capable of broad abstraction. At its most basic level, this third element enables man to acquire, process, store, represent, and continually re-acquire knowledge which is external to that man's subjective existence. This is calculation in the strictest sense. And it is the third element -- the strength, speed, and breadth of the predictive function -- which is synonymous with the word "intelligence." Higher animals have all three elements, but they're pretty hazy -- especially the third. And, in humans, short time horizons are synonymous with intellectual dullness. All of this is to say that if you have a "prediction machine" you're 90% of the way to a true "intelligence machine." It also, I think, suggests routes that might lead to more robust AI in the future. (Ground the AI, give it a limited physical presence in time and space, match its clocks to the outside world.)

quonn 1y ago
"Prediction" is hardly more than another term for inference. It's the very essence of machine learning. There is nothing new or useful in this concept.
- A_D_E_P_T 1y ago
  Point is that it's also exactly analogous to human intelligence. There's almost nothing else to it.
  - devmor 1y ago
    This is how you spot hype nonsense - claims that anything is analogous to human intelligence. Even absent all other objections, we don't understand the human mind well enough to make a claim like that.

cubefox 1y ago

Another difference is that you are predicting future sensory experiences in real-time, while LLMs "predict" text which a "helpful, honest, harmless" assistant would produce.

somewhereoutth 1y ago

> you build a world model The foundational conceit (if you will) of LLMs is that they build a semantic (world) model to 'make sense' of their training. However it is much more likely that they are simply building a syntactic model in response to the training. As far as I know there is no evidence of a semantic model emerging.

jebarker 1y ago
Maybe I don’t have a precise enough definition of syntax and semantics, but it seems like it’s more than just syntactic since interchangeable tokens in the same syntax affect the semantics of the sentence. Or do you view completing a prompt such as “The president of the United States is?” as a syntax question?
ijk 1y ago
There's some evidence of valid relationships: you can build a map of Manhattan by asking about directions from each street corner and plotting the relations. This is still entirely referential, but in a way that a human would see some relation to the actual thing, albeit in a somewhat weird and alien way.
deleted 1y ago
[deleted]
IanCal 1y ago
Is this not addressed by othellogpt?

layer8 1y ago

Lossy is an incomplete characterization. LLMs are also much more fluctuating and fuzzy. You can get wildly varying output depending on prompting, for what should be the same (even if lossy) knowledge. There is not just loss during the training, but also loss and variation during inference. An LLM overall is a much less coherent and consistent thing than most humans, in terms of knowledge, mindset, and elucidations.

NoMoreNicksLeft 1y ago

Imagine having the world's most comprehensive encyclopedia at your literal fingertips, 24 hours a day, but being so lazy that you offload the hard work of thinking by letting retarded software pathologically lie to you and then blindly accepting the non-answers it spits at you rather than typing in two or three keywords to Wikipedia and skimming the top paragraph. >I prefer to think of LLMs as lossy predictors. I've started to call them the Great Filter. In the latest issue of the comic book Lex Luthor attempts to exterminate humanity by hacking the LLM and having it inform humanity that they can hold their breath underwater for 17 hours.