3 ms·

I have to disagree with that. Like maybe for a toy example to demonstrate what I'm talking about, imagine I was teaching you the addition operation mod 100 and

I have to disagree with that. Like maybe for a toy example to demonstrate what I'm talking about, imagine I was teaching you the addition operation mod 100 and I gave a description of the operation f(x, y) = x + y % 100 for x, y in Z_100. If you take more than 100^2 samples to learn the function, I'm not sure you understand the function. Obviously, in that many samples, you could've just specified a look up table without understanding what each operation is doing or what the underlying domain is. Part of why sample efficiency is interesting is that humans have high sample efficiency since they somehow perform reasoning and this generalizes well to some pretty abstract spaces. As someone who's worked with ML models, I'm genuinely envious of the generalization capabilities of humans and I think it's something that researchers are going to have to work on. I'm pretty sure there's still a lot of skepticism in academia that scale is everything needed to achieve better models and that we're still missing lots of things. Some of my skepticism around claims of LLMs reasoning or performing human like things is that they really appear to not generalize well. Lots of the incredible examples people have shown are very slightly out of the bounds of the internet. When you start asking it for hard logic or to really synthesize something novel outside the domain of the internet, it rapidly begins to fail seemingly in proportion to the amount of knowledge the internet may have on it. How might we differentiate being a really good soft/fuzzy lookup table of the internet that is able to fuzzily mix language together from genuine knowledge and generalization. This might just be a testament to the sheer scope and size of the internet in how much apparent capabilities GPT has. This isn't to say they cannot be useful ever, a lot of work is derivative, but I think there's a large portion of the claim that it's understanding things that's unwarranted. Last I checked, chatGPT was giving wrong answers for the sums of very large numbers which is unusual if it understands addition.

HervalFreire 4y ago
You're describing over fitting to some look up table. Can't be what's happening here. Because the examples LLMs are answering are well out of bounds of the "100^2" training data. The internet is huge but it's not that huge. One can easily find chatGPT saying, doing or creating things that obviously come from a generalized model. It's actually trivial to find examples of chatGPT answering questions with responses that are wholly unique and distinct from the training data, as in the answer it gave you could not have existed anywhere on the internet. Clearly humans don't need that much training data. We can form generalizations from a much smaller sample size. This does not indicate that for machine learning a generalization doesn't exist in LLMs when clearly the answers demonstrate that it does.
- inkysigma 4y ago
  Like yes to some extent there is a mild amount of generalization in that it is not literally regurgitating the internet and it to some extent mixes text really well but I don't think that's obviously the full on generalization of understanding that humans have. These models obviously are more sample efficient at learning relationships than a literal lookup table but like I've already said: my example was obviously extreme for the purposes of illustration that sample efficiency does seem to matter. If you used 100^2 - 1 samples, I'm still not confident you truly understand the concept. However, if you use 5 samples: I'm pretty sure you've generalized so I was hoping to illustrate a gradient. I want to reemphasize another portion of my comment: it really does seem that when you step outside of the domain of the internet, the error rates rise dramatically especially when there is completely no analogous situation. Furthermore, the further from the internet samples, the seemingly more likely the error which should not occur if it understood these concepts for the purposes of generalization. Do you have links to examples you'd be willing to discuss? Many examples I see are directly one of the top results on Google. The more impressive ones mix multiple results with some coherency. Sometimes people ask for something novel but there's a weirdly close parallel on the internet. For example, people thought Sumplete was a new game but it turned out to be derivative: https://www.neowin.net/news/chatgpt-made-a-browser-puzzle-game-on-its-own-kind-of-and-you-can-play-it-right-now/ https://www.neowin.net/news/chatgpt-made-a-browser-puzzle-ga... I think this isn't as impressive at least towards generalization. It seems to stitch concepts pretty haphazardly like in the novel language above that doesn't seem to respect the description (after all, why use brackets in a supposedly indentation based language). However, many languages do use brackets. It seems to suggest it correlates probable answers rather than reasons.
  - HervalFreire 4y ago
    >I want to reemphasize another portion of my comment: it really does seem that when you step outside of the domain of the internet, the error rates rise dramatically especially when there is completely no analogous situation. This is not surprising. A human would suffer from similar errors at a similar rate if it were exclusively fed an interpretation of reality that only consisted of text from the internet. >These models obviously are more sample efficient at learning relationships than a literal lookup table but like I've already said: my example was obviously extreme for the purposes of illustration that sample efficiency does seem to matter. If you used 100^2 - 1 samples, Even within the context of the internet there are enough conversational scenarios where you can have chatGPT answer things in ways that are far more generalized then "minor". Take for example: https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ Read it to the end. In the beginning you could say that the terminal emulation does exist as a similar copy in some form on the internet. But the structure that was built in the end is unique enough that it could be said nothing like it has ever existed on the internet. Additionally you have to realize that while bash commands and results do exist in ON the internet, chatGPT cannot simply copy the logic and interactive behavior of the terminal from text. In order to do what it did (even in the beginning) it must "understand" what a shell is and it has to derive that understanding from internet text.

HervalFreire 4y ago

You're describing over fitting to some look up table. Can't be what's happening here. Because the examples LLMs are answering are well out of bounds of the "100^2" training data. The internet is huge but it's not that huge. One can easily find chatGPT saying, doing or creating things that obviously come from a generalized model. It's actually trivial to find examples of chatGPT answering questions with responses that are wholly unique and distinct from the training data, as in the answer it gave you could not have existed anywhere on the internet. Clearly humans don't need that much training data. We can form generalizations from a much smaller sample size. This does not indicate that for machine learning a generalization doesn't exist in LLMs when clearly the answers demonstrate that it does.

inkysigma 4y ago
Like yes to some extent there is a mild amount of generalization in that it is not literally regurgitating the internet and it to some extent mixes text really well but I don't think that's obviously the full on generalization of understanding that humans have. These models obviously are more sample efficient at learning relationships than a literal lookup table but like I've already said: my example was obviously extreme for the purposes of illustration that sample efficiency does seem to matter. If you used 100^2 - 1 samples, I'm still not confident you truly understand the concept. However, if you use 5 samples: I'm pretty sure you've generalized so I was hoping to illustrate a gradient. I want to reemphasize another portion of my comment: it really does seem that when you step outside of the domain of the internet, the error rates rise dramatically especially when there is completely no analogous situation. Furthermore, the further from the internet samples, the seemingly more likely the error which should not occur if it understood these concepts for the purposes of generalization. Do you have links to examples you'd be willing to discuss? Many examples I see are directly one of the top results on Google. The more impressive ones mix multiple results with some coherency. Sometimes people ask for something novel but there's a weirdly close parallel on the internet. For example, people thought Sumplete was a new game but it turned out to be derivative: https://www.neowin.net/news/chatgpt-made-a-browser-puzzle-game-on-its-own-kind-of-and-you-can-play-it-right-now/ https://www.neowin.net/news/chatgpt-made-a-browser-puzzle-ga... I think this isn't as impressive at least towards generalization. It seems to stitch concepts pretty haphazardly like in the novel language above that doesn't seem to respect the description (after all, why use brackets in a supposedly indentation based language). However, many languages do use brackets. It seems to suggest it correlates probable answers rather than reasons.
- HervalFreire 4y ago
  >I want to reemphasize another portion of my comment: it really does seem that when you step outside of the domain of the internet, the error rates rise dramatically especially when there is completely no analogous situation. This is not surprising. A human would suffer from similar errors at a similar rate if it were exclusively fed an interpretation of reality that only consisted of text from the internet. >These models obviously are more sample efficient at learning relationships than a literal lookup table but like I've already said: my example was obviously extreme for the purposes of illustration that sample efficiency does seem to matter. If you used 100^2 - 1 samples, Even within the context of the internet there are enough conversational scenarios where you can have chatGPT answer things in ways that are far more generalized then "minor". Take for example: https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ Read it to the end. In the beginning you could say that the terminal emulation does exist as a similar copy in some form on the internet. But the structure that was built in the end is unique enough that it could be said nothing like it has ever existed on the internet. Additionally you have to realize that while bash commands and results do exist in ON the internet, chatGPT cannot simply copy the logic and interactive behavior of the terminal from text. In order to do what it did (even in the beginning) it must "understand" what a shell is and it has to derive that understanding from internet text.