7 ms·

That’s basically the claim of the “Blurry JPEG of the Web” article. I agree that to a large degree that’s probably what’s going on here. But it’s not all there

by jhrmnn 3y ago

That’s basically the claim of the “Blurry JPEG of the Web” article. I agree that to a large degree that’s probably what’s going on here. But it’s not all there is to it. Large language models apparently can build some world models. See Othello-GPT for a proof of principle. In other words, once you “compress” to thoughts, there actually can be some degree of actual reasoning on “decompression”.

deleted 3y ago
[deleted]
mjburgess 3y ago
> Othello-GPT Has any one written the obvious reply to this paper yet? It's false. It's not true. The rules and the board state it "learns" are just functions of the token positions *by construction*; it is given the token positions. obviously it learns patterns in those positions. The paper is such a misfire, it's absurd. The problem with using a formal system to investigate this issue is that by construction the "distributional hypothesis" is true for that system. Ie., the pattern in distribution of the moves are (a very good model of) the rules and board-state. But this is clearly untrue for non-formal systems not constructed this way. Eg., text tokens are not distributed like causal structures in the world -- and it's absurd to suppose so. The distributional hypothesis is obviously false for almost every measurement system of interest: measures are not distributed like their causes. We invent measurement systems to encode information, not to be isomorphic to world-structure.
pmoriarty 3y ago
"That’s basically the claim of the “Blurry JPEG of the Web” article. I agree that to a large degree that’s probably what’s going on here." I strongly disagree with that view, and encourage reading the "Sparks of Artificial General Intelligence" paper[1] and watching the associated video[2], then just experiment with GPT4 using novel information that has been created after its training data cutoff and you'll see it reason about it. It can also output novel creative works about things that didn't exist in its training data. Sure, there's some "compression" going on in LLMs, but that's far from everything that's going on. [1] - https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712 [2] - https://m.youtube.com/watch?v=qbIk7-JPB2c https://m.youtube.com/watch?v=qbIk7-JPB2c

by jhrmnn 3y ago

deleted 3y ago

[deleted]

mjburgess 3y ago

> Othello-GPT Has any one written the obvious reply to this paper yet? It's false. It's not true. The rules and the board state it "learns" are just functions of the token positions *by construction*; it is given the token positions. obviously it learns patterns in those positions. The paper is such a misfire, it's absurd. The problem with using a formal system to investigate this issue is that by construction the "distributional hypothesis" is true for that system. Ie., the pattern in distribution of the moves are (a very good model of) the rules and board-state. But this is clearly untrue for non-formal systems not constructed this way. Eg., text tokens are not distributed like causal structures in the world -- and it's absurd to suppose so. The distributional hypothesis is obviously false for almost every measurement system of interest: measures are not distributed like their causes. We invent measurement systems to encode information, not to be isomorphic to world-structure.

pmoriarty 3y ago

"That’s basically the claim of the “Blurry JPEG of the Web” article. I agree that to a large degree that’s probably what’s going on here." I strongly disagree with that view, and encourage reading the "Sparks of Artificial General Intelligence" paper[1] and watching the associated video[2], then just experiment with GPT4 using novel information that has been created after its training data cutoff and you'll see it reason about it. It can also output novel creative works about things that didn't exist in its training data. Sure, there's some "compression" going on in LLMs, but that's far from everything that's going on. [1] - https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712 [2] - https://m.youtube.com/watch?v=qbIk7-JPB2c https://m.youtube.com/watch?v=qbIk7-JPB2c