3 ms·
16k context sounds exciting. The day I can throw a whole book at it and ask it arbitrary questions about it will be great. With 16k we are getting into full art
by weinzierl 3y ago
16k context sounds exciting. The day I can throw a whole book at it and ask it arbitrary questions about it will be great. With 16k we are getting into full article realm and that is already incredibly useful.
Is there any open model with a similar context length? [I'm not talking about the dubious for long context fine-tuned LLaMA variants, I mean the real thing.]
- 037 3y agoNot open, but with Anthropic the limit is 100k now https://news.ycombinator.com/item?id=35904773 https://news.ycombinator.com/item?id=35904773
- 101011 3y agoTry looking into vector databases to solve that problem. You can chunk up a book and embed those partitions into a vector database. Then you can take a query and fuzzy match the most relevant documents in your vector database, then feed it back to open AI to resolve an answer. It's brilliant. Postgres has an extension to support indexing the vectors, and there are some other open source and turnkey solutions in the market as well.
- weinzierl 3y agoBrilliant, but not the same. I think both approaches have their place and are not mutually exclusive.
- 101011 3y agoThey are not the same, but your initial problem of throw a whole book at it and have OpenAI give you an answer is a demo that is solvable in 20 lines of langchain code when leveraging a vector DB. Demo here: https://www.youtube.com/watch?v=h0DHDp1FbmQ https://www.youtube.com/watch?v=h0DHDp1FbmQ (github code is linked within as well)