3 ms·
Do you support local LLMs? I could only see an ENV for OPENAI_API_KEY.
by circularfoyers 2y ago
Do you support local LLMs? I could only see an ENV for OPENAI_API_KEY.
- Manik_agg 2y agoHey another co-founder of Tegon here, currently we only support Open AI models but plan to add local models support with Olamma soon.
- hrpnk 2y agoThere is also COHERE_API_KEY for the vector storage & search: https://github.com/tegonhq/tegon/blob/158b54af8d6f7cf4195c615510976b6025baef13/server/src/modules/vector/vector.service.ts#L30 https://github.com/tegonhq/tegon/blob/158b54af8d6f7cf4195c61...
- ranger_danger 2y agollama.cpp supports being an OpenAI API server, you would just need to change the URL in the Tegon source.