3 ms·
A device or system that, given a set of tokens, predicts the next token purely based on the previous tokens and its own initial state. It is incapable of coming
by dmitrygr 2y ago
A device or system that, given a set of tokens, predicts the next token purely based on the previous tokens and its own initial state. It is incapable of coming up with a new token type or changing its own workings
- Grimblewald 2y agoOn top of your definition being a bit circular, it is also like saying humans can't come up with new things, because we can't come up with new characters when talking online. We're limited by the character-set we're given but that doesn't mean we can't conceive of new characters and more importantly, can't represent new characters using existing character-sets using some encoding scheme to represent that new character. To that end, lets check with chatGPT if it can come up with a new and useful concept for a token, using existing characters to encode for it. https://chatgpt.com/share/67bd04cf-1c50-8008-8606-ccac5df98931 https://chatgpt.com/share/67bd04cf-1c50-8008-8606-ccac5df989... And there you have it, novel constructed from available ones, with a genuine pragmatic and LLM specific use case.
- suspended_state 2y ago[dead]
- Lerc 2y agoSo if they can come up with their own internal representations is that sufficient? https://arxiv.org/abs/2410.05864 https://arxiv.org/abs/2410.05864 Or are you wanting them to use keys that are not on their keyboard?