I also built this for myself, and I really do suggest it as a good project to get a grounded idea of what models can handle. Mainly - tool calling support just

I also built this for myself, and I really do suggest it as a good project to get a grounded idea of what models can handle. Mainly - tool calling support just merged in llama.cpp (https://github.com/ggerganov/llama.cpp/pull/9639 https://github.com/ggerganov/llama.cpp/pull/9639) this week, and it's been a fun exercise to put local LLMs through the wringer to see how they do at handling it. It's been a mixture of "surprisingly well" and "really badly".