3 ms·
So if a mechanism was developed for the LLM to add more tokens over time, you would retract your objection?
by exe34 3y ago
So if a mechanism was developed for the LLM to add more tokens over time, you would retract your objection?
- notahacker 3y agoNope (afaik LLMs are already capable of generating novel tokens parsing novel tokens generated by [other] LLMs) The point is that humans and their ancestors had been self-replicating, long-running biochemical processes for millions of years before language (and related abstractions) were even a thing. So I don't see the impressive performance of token-prediction algorithms at associating tokens representing human abstraction with each other in ways humans discern as meaningful as offering much insight into human behaviour, never mind giving a complete explanation for how it happens. The claim that the way we navigate the world boils down to "just" predicting the next token feels even more extraordinary in the context of prediction being something we're pretty bad at.