4 ms·
Cosign, I feel like the longer context window is mostly a dud, especially above 16K tokens. I don't miss sweating the details on what exactly to fit into 4096,
by refulgentis 3y ago
Cosign, I feel like the longer context window is mostly a dud, especially above 16K tokens.
I don't miss sweating the details on what exactly to fit into 4096, but if I just use GPT-4, I'm left with the distinct impression the current context extension stuff is hacky, and something needs to be done in training to actually make use of it.
Google miraculously getting 1M context with perfect recall is more fuel for my "don't tell me, show me" fire when it comes to Google. They're batting roughly 0 for 2 years now
- dontupvoteme 3y agoThe main thing I'd like large context for is the ability to shove a ton of API/Python/etc documentation into the window for a given problem so it knows better what to do. In terms of the actual output it can be annoying if the convo goes on too long and it hits the window, but for technical things this hardly matters as you can just restart it - more of an RP issue as i understand.