4 ms·
Pretty sweet to get to run models locally and have more advanced usages like tool calling, excited to try it out
by koinedad 2y ago
Pretty sweet to get to run models locally and have more advanced usages like tool calling, excited to try it out
- jackmpcollins 2y agoIt's a great start but there's a little more work to do for full OpenAI API compatibility, namely streaming support and the tool_choice parameter. Making it fully compatible would allow it be swapped in directly to frameworks like langchain and magentic [1] that I am building). [1] https://github.com/jackmpcollins/magentic/issues/207 https://github.com/jackmpcollins/magentic/issues/207