3 ms·
This is wrong. Human thinking and speech isn't autoregressive like LLM inference.
by root_axis 3mo ago
This is wrong. Human thinking and speech isn't autoregressive like LLM inference.
- infinite_spin 3mo agowhile the how is different, the what has many parallels. E.g. both the brain and LLMs appear to learn distributions of representations, they both develop a hierarchy of those representations, both have early layers that process simple features, with later ones processing more abstract concepts, both predict missing information...
- root_axis 3mo agoThe post I responded to stated that the commenter was just a next-word-sayer, but that's wrong. The similarities you draw aren't really relevant to my reply.
- infinite_spin 3mo agono disrespect intended, however I think my response is relevant, because the broader topic here is whether LLMs and the human mind share similar functions. They both do in fact have a lot of overlapping features, and a fundamental one is predicting next-thing, be that a word, image, or otherwise.
- root_axis 3mo agoIt's not relevant. However, if you want to talk about a broader point, that's ok. > LLMs appear to learn distributions of representations, they both develop a hierarchy of those representations, both have early layers that process simple features, with later ones processing more abstract concepts, both predict missing information. This type of superficial comparison isn't very meaningful, it's trivial to liken anything to a human biology in this manner. A plane and a bird both use wings to produce lift, it doesn't then follow that a bird and a plane are meaningfully similar.
- infinite_spin 3mo ago> A plane and a bird both use wings to produce lift, it doesn't then follow that a bird and a plane are meaningfully similar. The use of Bernoulli's principle to achieve lift is a fundamental and meaningfully similar function of both airplane and bird wings. That functional similarity is well known. > This type of superficial comparison isn't very meaningful The comparisons I provided are fundamental to both the human mind and LLMs.. that's pretty darn relevant.. and whether you find that trivial or not is a matter of opinion.
- semiquaver 3mo agoDo you not say your words one-at-a-time like everyone else? Otherwise I can’t see how my comment is “wrong”
- tuvix 3mo agoEven if you could understand human cognition to the level required to say, confidently, that it’s done one word at a time, it’s likely not! Natural language is not a prerequisite for human intelligence, as evidenced by the fact that we went from primates to commenting on HN. Natural language is, however, a prerequisite for the existence of LLMs. It’s more similar to methods for storing and retrieving information, like the printing press or a database, than it is to a sentient being. That’s not to say that LLMs can’t do crazy things, because they already have. Our language can encode a whole lot of information, and it’s incredible that we’ve found a way to distill that so effectively.
- skeptic_ai 3mo agoDeepseek zero didn’t mix up all languages in something very efficient?
- tuvix 3mo agoInteresting thought but I assume a lot of samples in the training corpus are examples of translation between languages and the same text in different languages.
- iainmerrick 3mo agoEven if you could understand human cognition to the level required to say, confidently, that it’s done one word at a time, it’s likely not! I think they’re not talking about cognition, but about output: regardless of what may be happening inside your brain, ultimately one word at a time comes out of your mouth, right? And you can’t then unsay it. When you put it in those terms, LLMs are in exactly the same boat.
- artisin 3mo agoOnly one word at a time!?! It's time you embrace the way of the diffusion model and hazily refine your entire thought until it's coherent.
- deleted 3mo ago[deleted]