3 ms·
The issue with tokens is that they shoot up inference memory usage
by avereveard 4y ago
The issue with tokens is that they shoot up inference memory usage
- muttled 4y agoOnce this barrier is broken down we'll see a lot of cool things. 32k on GPT-4 is already pretty cool but once we get into hundreds of thousands/millions of tokens of context we'll be able to easily do things only currently achievable with fine tuning and "memory" tricks. Assistants that remember everything you've ever told them, asking detailed questions about large datasets, even complex systems that are bootstrapped from the context.
- deleted 4y ago[deleted]