6 ms·

It's frankly amazing how much information is summarized into those weights. You do need to take whatever it says with a very healthy pinch of salt. It doesn't

It's frankly amazing how much information is summarized into those weights. You do need to take whatever it says with a very healthy pinch of salt. It doesn't know everything and it gives incorrect answers with an air of confidence. Here's an example: > who preceded emperor Commodus? The first emperor of Rome was Nerva (98-100 AD). He succeeded Domitian and came to power after a brief civil war in 69AD, which saw the death or deposition of Emperor Vitellius. On the one hand, that's just factually wrong. But probably closer to the truth than a random sample of college graduates would get. This is truly impressive.

pr337h4m 4y ago
The thesis behind the Hutter Prize was spot-on, except for focusing on lossless compression instead of lossy compression. http://prize.hutter1.net/ http://prize.hutter1.net/
- londons_explore 4y ago
  There is a mathematical mapping between lossy and lossless compression. They are effectively the same problem (theoretically, even though implentations are typically very different) LLM's would make excellent (but very slow) lossy or lossless compressors.
  - muyyatin2 4y ago
    Do you have some links/references for someone wanting to learn more about this?
    - optimalsolver 4y ago
      http://mattmahoney.net/dc/rationale.html http://mattmahoney.net/dc/rationale.html
    - telotortium 4y ago
      Here's a simple explanation: https://news.ycombinator.com/item?id=34726719 https://news.ycombinator.com/item?id=34726719. Basically, with a sufficiently good lossy compressor, you can use entropy coding to encode the difference between the decompression output and the original file.
- deleted 4y ago
  [deleted]
- modeless 4y ago
  Lossless is actually the appropriate test. These models output probabilities and can easily be turned into very effective lossless compressors with the use of arithmetic encoding. For a very interesting discussion see: https://www.youtube.com/watch?v=dO4TPJkeaaU https://www.youtube.com/watch?v=dO4TPJkeaaU The real problem with the Hutter prize is that the data and compute are too small. It seems obvious now that the regime where compression turns into AGI is far beyond the scale of the current contest (1 GB data, 50 CPU hours). If you simply bumped those limits up five or six orders of magnitude then the Hutter prize could be relevant today. But you'd also have to increase the size of the prize a few orders of magnitude because the cost of competing would be more than the current prize money. Of course a prize isn't really a necessary incentive to develop AGI, since getting an AGI is about the best incentive you could possibly imagine.
  - optimalsolver 4y ago
    >Lossless is actually the appropriate test Could you elaborate on why that is. It seems very non-obvious to me. In the realm of vision, after a certain point you would just be trying to compress noise that has nothing to do with the content of image/video.
    - modeless 4y ago
      That's OK. The purpose of the test is to compare approaches to compression. Noise in the dataset is a handicap but it affects all approaches, so it doesn't invalidate the test as a way to compare approaches. And true white noise is very rare, almost all real noise has characteristics that are at least somewhat compressible. And hopefully whoever constructs the dataset tries not to include tons of noise in it. Lossless compression is a great objective because it's impossible to cheat. When you do lossy compression you have to define a quality metric and as soon as you do that the game becomes cheating the quality metric rather than actually compressing the data in a useful way. I highly recommend watching the video I linked. Arithmetic encoding reduces the task of lossless compression to assigning probabilities for the next token, which is the exact task these models are trained to do.
    - inimino 4y ago
radarsat1 4y ago
I think a really interesting area of further research into LLMs is going to be how to further decouple "intelligence" from "knowledge". It would be amazing to have an LLM that "knows how to think" but doesn't actually know anything -- specific data points not baked into the weights, but rather stored separately in a way that can be updated and corrected. Perhaps a step forward is the "toolformer" router, making the LLM recognize when it is referencing a fact and have it look that up (use a "tool") instead of trying to infer it. I realize that even semantically this is kind of a fine line, hard facts vs inferences, so there's some philosophy to ponder here. What can be 'stored' and what can be 'calculated'. What kind of knowledge is innate to intelligent reasoning and what kind acts more like a lookup table. But, exactly because we can't easily define these two categories, I have a feeling that if LLMs are trained to actually learn by themselves how to distinguish them in a "fuzzy" way, we might start getting somewhere.
- Y_Y 4y ago
  If you want something that "knows how to think" then I don't think you'll find it in an LLM. You can't divorce knowledge from this kind of model, it guesses what word comes next based on its training on a corpus of knowledge.
  - georgelyon 4y ago
    No, but you could build something that guesses the next step in a list of steps to access knowledge from an external source.
  - gamegoblin 4y ago
    One could imagine training a much smaller LLM on synthetic data that didn't contain much or any real knowledge, but just examples of in-context reasoning. E.g. the training data might look like "a fooble is to a tooble as a fabble is to a" with the answer "tabble". So you feed it tons of these kind of nonsense training data that forces it to only learn the in-context reasoning part of language, and none of the world knowledge. That said, it is unclear to me how much value such a model would be. You could imagine a middle ground where it does have basic knowledge, such as dictionary definitions and things derivable from those. E.g. "A tree has leaves. Leaves are green. Therefore a tree has a part that is green." type stuff. So you could give it some amount of world-grounding and common sense knowledge, but nothing involving history, proper nouns, etc. I imagine you could make such a model much smaller than these giant LLMs.
up2isomorphism 4y ago
“Closed to truth” is actually the worst property of LLM models. Because it hard for human to detect if it is wrong or not.
- Fomite 4y ago
  Turns humans from writers into fact checkers, which it turns out is a harder skill.
  - up2isomorphism 4y ago
    It is actually much harder skill. And what worse is that any successful check makes the future checks harder. But there might be no significant improvement on the quality, just the difficulty of detecting the fallacy.
  - zamnos 4y ago
    What sort of facts are you thinking of? There are plenty of surface level facts that another model or a human should easily be able to verify as true or not. Was President Obama the 41st president of the United States, does 2+2 = 5, was Elvis the son of an artist, etc
    - Fomite 4y ago
      In my sort of informal "How would I like to use this?" testing, this ranges from easily verifiable facts (i.e. "Did you get where I got my degree right?") to structural misunderstandings of important concepts. The former would be easy enough to fix - though it undermines the utility and if not caught could carry ramifications for people, but the latter genuinely concerns me.
- chii 4y ago
  But if you create an adversarial ai to do fact checking - aka, a reward function where if a fact is found on the internet that contradicts the given GPT response, then would it work?
Fomite 4y ago
This is one of the things that LLMs seem to be struggling with. They're very good at tone, and feeling right without actually being write. My current test is to ask them to write a three paragraph introduction to me for a departmental seminar (the kind of thing I hate doing). Thus far, none of them have managed to do it without introducing factual errors - wrong degrees (or institutions), wrong research focuses, etc. But they're all credible sounding in their wrongness.
- bleuchase 4y ago
  > They're very good at tone, and feeling right without actually being write. The text they generate is probably free of errors like this one as well
  - Fomite 4y ago
    Probably.
    - bleuchase 4y ago
      I’m sure someone is working on adding support for human-like errors in LLM outputs.
- crucialfelix 4y ago
  Write a crappy intro for yourself with all the facts included. Ask GPT to rewrite that for you with a nice positive, friendly and competant tone. It's really good at that.

pr337h4m 4y ago

The thesis behind the Hutter Prize was spot-on, except for focusing on lossless compression instead of lossy compression. http://prize.hutter1.net/ http://prize.hutter1.net/

londons_explore 4y ago
There is a mathematical mapping between lossy and lossless compression. They are effectively the same problem (theoretically, even though implentations are typically very different) LLM's would make excellent (but very slow) lossy or lossless compressors.
- muyyatin2 4y ago
  Do you have some links/references for someone wanting to learn more about this?
  - optimalsolver 4y ago
    http://mattmahoney.net/dc/rationale.html http://mattmahoney.net/dc/rationale.html
  - telotortium 4y ago
    Here's a simple explanation: https://news.ycombinator.com/item?id=34726719 https://news.ycombinator.com/item?id=34726719. Basically, with a sufficiently good lossy compressor, you can use entropy coding to encode the difference between the decompression output and the original file.
deleted 4y ago
[deleted]
modeless 4y ago
Lossless is actually the appropriate test. These models output probabilities and can easily be turned into very effective lossless compressors with the use of arithmetic encoding. For a very interesting discussion see: https://www.youtube.com/watch?v=dO4TPJkeaaU https://www.youtube.com/watch?v=dO4TPJkeaaU The real problem with the Hutter prize is that the data and compute are too small. It seems obvious now that the regime where compression turns into AGI is far beyond the scale of the current contest (1 GB data, 50 CPU hours). If you simply bumped those limits up five or six orders of magnitude then the Hutter prize could be relevant today. But you'd also have to increase the size of the prize a few orders of magnitude because the cost of competing would be more than the current prize money. Of course a prize isn't really a necessary incentive to develop AGI, since getting an AGI is about the best incentive you could possibly imagine.
- optimalsolver 4y ago
  >Lossless is actually the appropriate test Could you elaborate on why that is. It seems very non-obvious to me. In the realm of vision, after a certain point you would just be trying to compress noise that has nothing to do with the content of image/video.
  - modeless 4y ago
    That's OK. The purpose of the test is to compare approaches to compression. Noise in the dataset is a handicap but it affects all approaches, so it doesn't invalidate the test as a way to compare approaches. And true white noise is very rare, almost all real noise has characteristics that are at least somewhat compressible. And hopefully whoever constructs the dataset tries not to include tons of noise in it. Lossless compression is a great objective because it's impossible to cheat. When you do lossy compression you have to define a quality metric and as soon as you do that the game becomes cheating the quality metric rather than actually compressing the data in a useful way. I highly recommend watching the video I linked. Arithmetic encoding reduces the task of lossless compression to assigning probabilities for the next token, which is the exact task these models are trained to do.
  - inimino 4y ago

radarsat1 4y ago

I think a really interesting area of further research into LLMs is going to be how to further decouple "intelligence" from "knowledge". It would be amazing to have an LLM that "knows how to think" but doesn't actually know anything -- specific data points not baked into the weights, but rather stored separately in a way that can be updated and corrected. Perhaps a step forward is the "toolformer" router, making the LLM recognize when it is referencing a fact and have it look that up (use a "tool") instead of trying to infer it. I realize that even semantically this is kind of a fine line, hard facts vs inferences, so there's some philosophy to ponder here. What can be 'stored' and what can be 'calculated'. What kind of knowledge is innate to intelligent reasoning and what kind acts more like a lookup table. But, exactly because we can't easily define these two categories, I have a feeling that if LLMs are trained to actually learn by themselves how to distinguish them in a "fuzzy" way, we might start getting somewhere.

Y_Y 4y ago
If you want something that "knows how to think" then I don't think you'll find it in an LLM. You can't divorce knowledge from this kind of model, it guesses what word comes next based on its training on a corpus of knowledge.
- georgelyon 4y ago
  No, but you could build something that guesses the next step in a list of steps to access knowledge from an external source.
- gamegoblin 4y ago
  One could imagine training a much smaller LLM on synthetic data that didn't contain much or any real knowledge, but just examples of in-context reasoning. E.g. the training data might look like "a fooble is to a tooble as a fabble is to a" with the answer "tabble". So you feed it tons of these kind of nonsense training data that forces it to only learn the in-context reasoning part of language, and none of the world knowledge. That said, it is unclear to me how much value such a model would be. You could imagine a middle ground where it does have basic knowledge, such as dictionary definitions and things derivable from those. E.g. "A tree has leaves. Leaves are green. Therefore a tree has a part that is green." type stuff. So you could give it some amount of world-grounding and common sense knowledge, but nothing involving history, proper nouns, etc. I imagine you could make such a model much smaller than these giant LLMs.

up2isomorphism 4y ago

“Closed to truth” is actually the worst property of LLM models. Because it hard for human to detect if it is wrong or not.

Fomite 4y ago
Turns humans from writers into fact checkers, which it turns out is a harder skill.
- up2isomorphism 4y ago
  It is actually much harder skill. And what worse is that any successful check makes the future checks harder. But there might be no significant improvement on the quality, just the difficulty of detecting the fallacy.
- zamnos 4y ago
  What sort of facts are you thinking of? There are plenty of surface level facts that another model or a human should easily be able to verify as true or not. Was President Obama the 41st president of the United States, does 2+2 = 5, was Elvis the son of an artist, etc
  - Fomite 4y ago
    In my sort of informal "How would I like to use this?" testing, this ranges from easily verifiable facts (i.e. "Did you get where I got my degree right?") to structural misunderstandings of important concepts. The former would be easy enough to fix - though it undermines the utility and if not caught could carry ramifications for people, but the latter genuinely concerns me.
chii 4y ago
But if you create an adversarial ai to do fact checking - aka, a reward function where if a fact is found on the internet that contradicts the given GPT response, then would it work?

Fomite 4y ago

This is one of the things that LLMs seem to be struggling with. They're very good at tone, and feeling right without actually being write. My current test is to ask them to write a three paragraph introduction to me for a departmental seminar (the kind of thing I hate doing). Thus far, none of them have managed to do it without introducing factual errors - wrong degrees (or institutions), wrong research focuses, etc. But they're all credible sounding in their wrongness.

bleuchase 4y ago
> They're very good at tone, and feeling right without actually being write. The text they generate is probably free of errors like this one as well
- Fomite 4y ago
  Probably.
  - bleuchase 4y ago
    I’m sure someone is working on adding support for human-like errors in LLM outputs.
crucialfelix 4y ago
Write a crappy intro for yourself with all the facts included. Ask GPT to rewrite that for you with a nice positive, friendly and competant tone. It's really good at that.