24 ms·
Why would anyone need this much context? Genuine question. It's not worth the drop in quality IMO.
by hnipps 7mo ago
Why would anyone need this much context? Genuine question. It's not worth the drop in quality IMO.
- tommek4077 7mo agoYou sound like: "Why would anyone need more than 640KB RAM?!".
- hnipps 7mo agoThat’s just not comparable. I’ll use your figure: If you use 400KB RAM for a process, using the remaining 240KB for something else doesn’t degrade the performance of the initial process (assuming nothing is trying to use more than the available RAM). Each unit of RAM is independent no? Every token of context you use causes a drop in LLM performance. More RAM == more processes can be supported with no degradation More context != more stuff can be done with no degradation