3 ms·
Context extension methods exist and work. Please educate yourself about these rather than confidentially saying wrong things.
by Der_Einzige 1y ago
Context extension methods exist and work. Please educate yourself about these rather than confidentially saying wrong things.
- WhitneyLand 1y agoNot sure what you’re disagreeing with? Context window size limits are not artificial. It takes real time/money/resources to increase them. There are a few ways to approach the problem. Pre-training on longer context lengths I’ve already mentioned. Fine-tuning techniques (like LongRoPE) I’ve already mentioned. Inference time context extension tricks I didn’t mention because the papers I’ve seen seem to suggest there’s often problems with quality or unfavorable tradeoffs. There’s no magic way around these limits, it’s a real engineering problem.