3 ms·
The fundamental techniques that they use are highly lossey and are far inferior to ultra-long context length models where you can do it all in one prompt. Hate
by Der_Einzige 4y ago
The fundamental techniques that they use are highly lossey and are far inferior to ultra-long context length models where you can do it all in one prompt. Hate to break it to you and all the others.
- nico 4y agoWhere can someone find and try ultra-long context length models? Any links?
- intelVISA 4y agoThe longest one that is generally available is always going to be yourself :)
- thelittleone 4y agoMy context model is getting shorter and fuzzier.
- EntrePrescott 4y ago… but still the weights are increasing ;)
- hombre_fatal 4y ago> Hate to break it to you and all the others. Jeez. Their comment is quite obviously a complementary one in response to the limitation rather than a corrective one about the limitation.
- faizshah 4y agoThe methods they employ are to improve the context being given to the model irrespective of the context length. Even when the context length improves these methods will be used to decrease the search space and resources required for a single task (think about stream search vs indexed search). I’m also curious what paper you are referencing that finds that more context vs more relevant context yields better results? A good survey of the methods for “Augmented Language Models” (CoT, etc.) is here: https://arxiv.org/pdf/2302.07842.pdf https://arxiv.org/pdf/2302.07842.pdf