3 ms·
I'm still trying to figure out the answer to that question myself. The absolute easiest approach right now is to use Claude, since it has a 100,000 token limit
by simonw 3y ago
I'm still trying to figure out the answer to that question myself.
The absolute easiest approach right now is to use Claude, since it has a 100,000 token limit - so you can stuff a ton of documentation into it at once and start asking questions.
Doing RAG with smaller models requires much more cleverness, which I'm only just starting to explore.
- stavros 3y agoThat's fair, thanks! Do you plan to integrate the cleverness into LLM, so we can benefit from it too? I'm not sure if LLM can be used as a library, currently, I've only been using it as a cli, but it would be great if I could use it in my programs without shelling out.
- simonw 3y agoYes, all of this stuff will end up in LLM - either in core or as a plugin for it. LLM works as a library already, but there's definitely room for improvement there: https://llm.datasette.io/en/stable/python-api.html https://llm.datasette.io/en/stable/python-api.html https://llm.datasette.io/en/stable/embeddings/python-api.html https://llm.datasette.io/en/stable/embeddings/python-api.htm...
- stavros 3y agoThis is great news, thanks!