2 ms·
Yeah, I think that we need to probe how much these models are merely memorising their inputs rather than learning a more genereal, compressed (sortof) represent
by disgruntledphd2 4y ago
Yeah, I think that we need to probe how much these models are merely memorising their inputs rather than learning a more genereal, compressed (sortof) representation.
Honestly though, ten years ago NLP was using Bag of Words and Markov Chains, it's pretty phenomenal how far it's come.
To be fair, I mostly agree with your main point, but it's still remarkably impressive progress in a relatively short period of time.
- pphysch 4y agoIt is impressive and useful technology but Gates is misleading by implying that the LLM is doing "critical thinking", when it is doing the opposite -- shallow regurgitation of "information" (i.e. tokens).
- medvezhenok 4y agoExperiments have demonstrated that GPT-4 does have a basic theory of mind, so it does not simply regurgitate.
- disgruntledphd2 4y agoNo, experiments have demonstrated that GPT-4 can respond as if it had theory of mind. Basically all measures of psychological constructs need to be rethought given the development of LLMs.