2 ms·
OK, so this is one of those things where the query first goes to some kind of lookup system, and you get back something which is fed into the LLM as part of the
by Animats 3y ago
OK, so this is one of those things where the query first goes to some kind of lookup system, and you get back something which is fed into the LLM as part of the prompt.
Is the lookup running locally (not on OpenAI)? Can you look at the output of the lookup to see what hopefully relevant info it is throwing into the LLM? Can you use this with some local open-source LLM system?
- ryoshu 3y agoYou can run it locally depending on the machine (13b models will run on a 32GB Apple Silicon laptop). You can also spin up one of the larger models on Huggingface or similar. If you want to productize it you'll likely need devops/mlops and money.