4 ms·
llama.cpp still doesn't support gpt-oss tool calling. https://github.com/ggml-org/llama.cpp/pull/15158 https://github.com/ggml-org/llama.cpp/pull/15158 (among o
by mhitza 1y ago
llama.cpp still doesn't support gpt-oss tool calling. https://github.com/ggml-org/llama.cpp/pull/15158 https://github.com/ggml-org/llama.cpp/pull/15158 (among other similar PRs)
But I also couldn't get vllm, or transformers serve, or ollama (400 response on /v1/chat/completions) working today with gpt-oss. OpenAI's cookbooks aren't really copy paste instructions. They probably tested on a single platform with preinstalled python packages which they forgot to mention :))
- refulgentis 1y agoRe: gpt-oss tool calls support, I don't think that's true, I've been using it for days. Then again, I did write my own harmony parser...(Noting for audience as you imply, neither does Ollama. Thing here is you either gotta hope all your users have nicely formed templates in their ggufs (they do not) or sometimes step in to ex. here, note the OpenAI chat completions-alike API llama.cpp provides will output a text response that you'll need to parse into a tool call yourself, until they implement a harmony parser)
- electroglyph 1y agogpt-oss are still being actively fixed right this moment, and there have already been quite a few fixes.