2 ms·

One could imagine training a much smaller LLM on synthetic data that didn't contain much or any real knowledge, but just examples of in-context reasoning. E.g.

One could imagine training a much smaller LLM on synthetic data that didn't contain much or any real knowledge, but just examples of in-context reasoning. E.g. the training data might look like "a fooble is to a tooble as a fabble is to a" with the answer "tabble". So you feed it tons of these kind of nonsense training data that forces it to only learn the in-context reasoning part of language, and none of the world knowledge. That said, it is unclear to me how much value such a model would be. You could imagine a middle ground where it does have basic knowledge, such as dictionary definitions and things derivable from those. E.g. "A tree has leaves. Leaves are green. Therefore a tree has a part that is green." type stuff. So you could give it some amount of world-grounding and common sense knowledge, but nothing involving history, proper nouns, etc. I imagine you could make such a model much smaller than these giant LLMs.

chii 4y ago
The reason, i suspect, that this method won't work, is because the training dataset has knowledge embedded in it, regardless of whether you think it only contains in-context logic. The evidence is that there are these "glitched" tokens : https://www.youtube.com/watch?v=WO2X3oZEJOA https://www.youtube.com/watch?v=WO2X3oZEJOA if you watched to the end of that video, it explores a very plausible reason why such glitched tokens exist - it is because the training dataset has a bunch of "nonsense" (a large dump of the subreddit that just counts a number, as well as various other "data"). So my hypothesis is that if you attempted the in-context logic training dataset, it would produce large number of glitched tokens (different ones, but equally as "illogical" to the user).
- gamegoblin 4y ago
  Tokenization is purely an implementation detail. If OpenAI had cared, they could have deleted those obviously glitched tokens from their tokenizer. They just didn't inspect it carefully and/or care. GPT4 does not suffer from the same glitched tokens as GPT3, presumably because it uses a different tokenizer. Furthermore, there are LLMs that operate on single bytes instead of multi-character tokens, totally obviating that problem.

chii 4y ago

The reason, i suspect, that this method won't work, is because the training dataset has knowledge embedded in it, regardless of whether you think it only contains in-context logic. The evidence is that there are these "glitched" tokens : https://www.youtube.com/watch?v=WO2X3oZEJOA https://www.youtube.com/watch?v=WO2X3oZEJOA if you watched to the end of that video, it explores a very plausible reason why such glitched tokens exist - it is because the training dataset has a bunch of "nonsense" (a large dump of the subreddit that just counts a number, as well as various other "data"). So my hypothesis is that if you attempted the in-context logic training dataset, it would produce large number of glitched tokens (different ones, but equally as "illogical" to the user).

gamegoblin 4y ago
Tokenization is purely an implementation detail. If OpenAI had cared, they could have deleted those obviously glitched tokens from their tokenizer. They just didn't inspect it carefully and/or care. GPT4 does not suffer from the same glitched tokens as GPT3, presumably because it uses a different tokenizer. Furthermore, there are LLMs that operate on single bytes instead of multi-character tokens, totally obviating that problem.