3 ms·
To support many other models you should look at ollama - it provides a REST API on your machine for local inference that works just like OpenAI
by ijustlovemath 3y ago
To support many other models you should look at ollama - it provides a REST API on your machine for local inference that works just like OpenAI
- danenania 3y agoThanks, I'm aware of ollama and the open source model ecosystem, but I haven't done a deep dive yet, so all the info in this thread has been quite helpful.
- ijustlovemath 3y agoIn theory, all you have to do is redirect the API gateway to localhost and all your existing integrations should just work!
- danenania 3y agoThere's an issue here to keep track of this: https://github.com/plandex-ai/plandex/issues/20 https://github.com/plandex-ai/plandex/issues/20 It seems that while ollama does have partial OpenAI API compatibility, it's missing function calling, so that's a blocker for now.