3 ms·
There are already some cool projects that help LLM go beyond the context window limitation and work with even larger codebases like https://github.com/jerryjliu
by int_is_compress 4y ago
There are already some cool projects that help LLM go beyond the context window limitation and work with even larger codebases like https://github.com/jerryjliu/llama_index https://github.com/jerryjliu/llama_index and https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain.
- Der_Einzige 4y agoThe fundamental techniques that they use are highly lossey and are far inferior to ultra-long context length models where you can do it all in one prompt. Hate to break it to you and all the others.
- nico 4y agoWhere can someone find and try ultra-long context length models? Any links?
- intelVISA 4y agoThe longest one that is generally available is always going to be yourself :)
- thelittleone 4y agoMy context model is getting shorter and fuzzier.
- EntrePrescott 4y ago… but still the weights are increasing ;)
- hombre_fatal 4y ago> Hate to break it to you and all the others. Jeez. Their comment is quite obviously a complementary one in response to the limitation rather than a corrective one about the limitation.
- faizshah 4y agoThe methods they employ are to improve the context being given to the model irrespective of the context length. Even when the context length improves these methods will be used to decrease the search space and resources required for a single task (think about stream search vs indexed search). I’m also curious what paper you are referencing that finds that more context vs more relevant context yields better results? A good survey of the methods for “Augmented Language Models” (CoT, etc.) is here: https://arxiv.org/pdf/2302.07842.pdf https://arxiv.org/pdf/2302.07842.pdf