4 ms·
Thank you for sharing this! I have one question: Is there any plan to add support for local LLM / embeddings models?
by metalwhale 2y ago
Thank you for sharing this! I have one question: Is there any plan to add support for local LLM / embeddings models?
- motoxpro 2y ago"Right now the system only supports OpenAI as an embedding provider, but we plan to extend with local and OSS model support soon." In the post you responded to
- metalwhale 2y agoHaha I feel so dumb now. Thank you!
- aargh_aargh 2y agoThis question keeps popping up but I don't get it. Everyone and their dog has an OpenAI-compatible API. Why not just serve a local LLM and put api.openai.com 127.0.0.1 in your hosts file? I mean why is that even a question? Is there some fundamental difference between the black box that is GPT-* and say, LLaMA, that I don't grok?
- deleted 2y ago[deleted]