5 ms·
isnt there a limit on prompt size? how would you actually use this? Im not very up to speed on this stuff
by Exuma 2y ago
isnt there a limit on prompt size? how would you actually use this? Im not very up to speed on this stuff
- lolinder 2y agoMost projects would be way too big to put into a prompt—even if technically you're within the official context window, those are often misleading—the actual window where input is actually useful is usually much smaller than advertised. What you can do with something like this is store it in a database and then query it for relevant chunks, which you then feed to the LLM as needed.
- jackstraw14 2y agoIdeally let the LLM chunk it up and figure out when to use those chunks.
- tom1337 2y agoI wonder if building a local version of this which resolves dependency paths of the file your currently working on to a certain level so the LLM gains more context of related files instead of just the whole repo (which could be insane if you use a monorepo)
- xnx 2y agoGemini Pro has a 2 million character context window which is ~1000 pages of code.