3 ms·

> I do not need the whole model. I don't need the model to have knowledge of the rain history of Algeria nor how many colors are in the Russian flag Isn’t the

> I do not need the whole model. I don't need the model to have knowledge of the rain history of Algeria nor how many colors are in the Russian flag Isn’t the performance gap between quantized and full models indicative that even if you aren’t using it directly, the model knowing the colors in the Russian flag does have something to do with the intelligence you demand?

KetoManx64 4mo ago
Do quantized models specifically prune out specific knowledge? I think they just compress things down but they're still in there. You'd most likely need to do that when you're doing the initial model training, but I'm not expert.
- JumpCrisscross 4mo ago
  > they just compress things down but they're still in there The compression is almost certainly in part specific knowledge getting fuzzed.
  - DennisP 4mo ago
    Yeah, but it's everything getting fuzzed, including the parts you care about.
    - JumpCrisscross 4mo ago
      Sure. There is a legitimate question around whether one can selectively excise “useless” knowledge. My guess is you can’t. The act of learning it encodes both the act of learning and the knowledge per se. The former is the power of the LLM. (I personally force mine to double check everything instead of going off memory.)
kibwen 4mo ago
Quantizing is one thing. But in general it's self-evident that training the model on information that is irrelevant to your use case does not necessarily improve ability, otherwise you'd have AGI just from reinforcing your model on memorizing the first 10^50 digits of pi. Likewise, LLMs do not violate the laws of information theory, and therefore the only way to encode X amount of information in Y amount of bits where X > Y is by performing what is effectively lossy compression, and as X grows larger relative to Y the compression ratio must change to lose ever more information. Yes, for the sake of making chatbots that are "conversational" in that they can interpret natural language as input and produce code as output you can easily benefit in incidental and unintuitive ways by training it on more natural language text. But for a given fixed parameter size, it's possible to produce a better model for a specific task by selectively not muddying its training set in the first place with things that are likely irrelevant to the task.
- tiahura 4mo ago
  Apparently irrelevant data can help because model weights are entangled.
- coldtea 4mo ago
  >But in general it's self-evident that training the model on information that is irrelevant to your use case does not necessarily improve ability, otherwise you'd have AGI just from reinforcing your model on memorizing the first 10^50 digits of pi. It's hardly self-evident, and your counter-example is hardly applicable. The first 10^50 of pi is not the same as having BREADTH of information in the training data, which is the whole point not just any random "information that is irrelevant to your use case". not to mention that the first 10^50 digits of pi compress to quite small formula, so not much information there to begin with from a shannon/kolmogorov perspective
  - kibwen 4mo ago
    It is self-evident. Bringing up Kolmogorov complexity is irrelevant, we're talking about rote memorization, but if you can't ignore the given example then replace "digits of pi" with "bits of output from a true random number generator". There's an infinite amount of information that we could shove into a model, and a finite amount of bits with which to store any of that information such that it can be usefully recalled or form useful logical associations.
    - coldtea 4mo ago
      "rote memorization" is not the right way to describe how an LLM works. The memorization of say 100000 world facts through training texts, which enrich model associations all around, is absolutely not the same as rote memorization on 10^50 digits of pi. Not for a human, and even more so, not for an LLM. An LLM trained with digits of pi and one trained with books and posts, even if they both have the exact same amount of bytes of training input, would not be comparable in any way in utility and reasoning capabilities. >There's an infinite amount of information that we could shove into a model, and a finite amount of bits with which to store any of that information such that it can be usefully recalled or form useful logical associations. Which is irrelevant. Anyway, the amount of information that doesn't form useful logical associations is even larger (e.g. actual human books vs possible permutations of characters and spaces). Just like those (random) possible permutations of characters aren't good for LLM input to get logical associations out of it, pi isn't either (logical associations of the kind we care for and expect, not of the kind related to pi's sequences). Also it's not only not self-evident, it's also apparently wrong.

KetoManx64 4mo ago

Do quantized models specifically prune out specific knowledge? I think they just compress things down but they're still in there. You'd most likely need to do that when you're doing the initial model training, but I'm not expert.

JumpCrisscross 4mo ago
> they just compress things down but they're still in there The compression is almost certainly in part specific knowledge getting fuzzed.
- DennisP 4mo ago
  Yeah, but it's everything getting fuzzed, including the parts you care about.
  - JumpCrisscross 4mo ago
    Sure. There is a legitimate question around whether one can selectively excise “useless” knowledge. My guess is you can’t. The act of learning it encodes both the act of learning and the knowledge per se. The former is the power of the LLM. (I personally force mine to double check everything instead of going off memory.)

kibwen 4mo ago

Quantizing is one thing. But in general it's self-evident that training the model on information that is irrelevant to your use case does not necessarily improve ability, otherwise you'd have AGI just from reinforcing your model on memorizing the first 10^50 digits of pi. Likewise, LLMs do not violate the laws of information theory, and therefore the only way to encode X amount of information in Y amount of bits where X > Y is by performing what is effectively lossy compression, and as X grows larger relative to Y the compression ratio must change to lose ever more information. Yes, for the sake of making chatbots that are "conversational" in that they can interpret natural language as input and produce code as output you can easily benefit in incidental and unintuitive ways by training it on more natural language text. But for a given fixed parameter size, it's possible to produce a better model for a specific task by selectively not muddying its training set in the first place with things that are likely irrelevant to the task.

tiahura 4mo ago
Apparently irrelevant data can help because model weights are entangled.
coldtea 4mo ago
>But in general it's self-evident that training the model on information that is irrelevant to your use case does not necessarily improve ability, otherwise you'd have AGI just from reinforcing your model on memorizing the first 10^50 digits of pi. It's hardly self-evident, and your counter-example is hardly applicable. The first 10^50 of pi is not the same as having BREADTH of information in the training data, which is the whole point not just any random "information that is irrelevant to your use case". not to mention that the first 10^50 digits of pi compress to quite small formula, so not much information there to begin with from a shannon/kolmogorov perspective
- kibwen 4mo ago
  It is self-evident. Bringing up Kolmogorov complexity is irrelevant, we're talking about rote memorization, but if you can't ignore the given example then replace "digits of pi" with "bits of output from a true random number generator". There's an infinite amount of information that we could shove into a model, and a finite amount of bits with which to store any of that information such that it can be usefully recalled or form useful logical associations.
  - coldtea 4mo ago
    "rote memorization" is not the right way to describe how an LLM works. The memorization of say 100000 world facts through training texts, which enrich model associations all around, is absolutely not the same as rote memorization on 10^50 digits of pi. Not for a human, and even more so, not for an LLM. An LLM trained with digits of pi and one trained with books and posts, even if they both have the exact same amount of bytes of training input, would not be comparable in any way in utility and reasoning capabilities. >There's an infinite amount of information that we could shove into a model, and a finite amount of bits with which to store any of that information such that it can be usefully recalled or form useful logical associations. Which is irrelevant. Anyway, the amount of information that doesn't form useful logical associations is even larger (e.g. actual human books vs possible permutations of characters and spaces). Just like those (random) possible permutations of characters aren't good for LLM input to get logical associations out of it, pi isn't either (logical associations of the kind we care for and expect, not of the kind related to pi's sequences). Also it's not only not self-evident, it's also apparently wrong.