4 ms·
When an LLM can re-tokenize on the fly, due to newly learned data, let me know. It won't prove intelligence, but at least it won't be static like a book.
by b112 1y ago
When an LLM can re-tokenize on the fly, due to newly learned data, let me know.
It won't prove intelligence, but at least it won't be static like a book.
- hatthew 1y agohttps://x.com/sukjun_hwang/status/1943703574908723674 https://x.com/sukjun_hwang/status/1943703574908723674 > Tokenization has been the final barrier to truly end-to-end language models. > We developed the H-Net: a hierarchical network that replaces tokenization with a dynamic chunking process directly inside the model, automatically discovering and operating over meaningful units of data
- b112 1y agoGood to see. Will be better to see once the scope is clear. But something such as this is required to move towards actual intelligence.
- danielmarkbruce 1y agoThis is a dumb critique. A thin wrapper running a new samples of training data and updating weights is something already done in many situations. A sophisticated rag system incorporates new information even if the weights themselves aren't updated, effectively giving "new memory". LLMs have problems, in practice being "static" aint one of them.