3 ms·
No more "stochastic parrots" and "LLM's can never produce anything novel, just regurgitate" comments anymore huh?
by reducesuffering 2mo ago
No more "stochastic parrots" and "LLM's can never produce anything novel, just regurgitate" comments anymore huh?
- andriy_koval 2mo agoSure, looks like 5% of math can be solved by 1T parameters stochastic parrot after NN trillion attempts (burned tokens). By numbers it could be less impressive than some brute force distributed chess engine.. Prove me wrong.
- rcxdude 2mo agoIt's obviously significantly better than brute force. A trillion is absolutely minuscule compared to the size of the search space (like, 'rounding down to 1 is basically the same' small).
- andriy_koval 2mo agotrillion indicates that its still brute force and not some search based on solid intelligent ranking function. Also, I am not expert in this specific topic, but base on comments looks like claude parroted few recent works?
- 8note 2mo agoi still havent seen an llm invent new tokens/words over the course of a session? the novelty remains a structural problem with how they work and what the weights try to predict