3 ms·
What this shows, IMO, is that quite a lot of the expected output is almost literally present in the training data.
by tgv 3y ago
What this shows, IMO, is that quite a lot of the expected output is almost literally present in the training data.
- flawsofar 3y agoPhrases and ideas repeat, because the universe has patterns and structure.
- tgv 3y agoBut it also suggests that the neural networks are not the miracle as some proclaim. They get their knowledge from compressing a very large amount of text, but add little in terms of inference. Its success is then a function of its size, but that size cannot grow much. It also ties in with the idea that the current ANNs don't discriminate text, and repeat bad input as readily as anything else.
- naasking 2y agoCompressing nearly all of human knowledge into a desktop PC that can converse in natural language and translate between languages is the miracle.
- tgv 2y agoIt's miraculous, but it's not nearly nearly all of human knowledge. Ask anything slightly specialist, and it'll give wrong answers. ChatGPT's translation is also not well developed. Llama doesn't have any, I believe.