There are certain "quality of life" issues with the open source ecosystem. I don't blame them for choosing (not at all) OpenAI. For one putting together a chatg

There are certain "quality of life" issues with the open source ecosystem. I don't blame them for choosing (not at all) OpenAI. For one putting together a chatgpt like experience that supports reliably at least chat history and per user system prompts of different models, requires a chat client that doesn't exist (at least it didn't a couple of months ago when I was looking for one). The closest we have now is a vscode plugin written by one guy I had to modify to work with multiple models (but it is a pretty good OpenAI api client). Also, OpenAI API is a defacto standard for clients talking to AI chat bots. To set it up with Mixtral I had to put together a non trivial system of huggingface TGI server (code extended to support token bias, cfg and negative prompts), and a (very slightly modified) litellm proxy to translate OpenAI api to TGI API. These products were used from the latest github branch and all had various shortcomings that required coding to resolve. Now I can say I truly have an "OpenAI - like" chat experience. But one huge functionality is missing. Function calling. Although implementing it now that I already have context free grammars is not that difficult, it still requires time I haven't found yet. Compare this to just paying a fee every month and getting it all done for you. Still I believe it is very important people recreate what OpenAI offers locally using open source software. Why? Be it is clear AI's like chatgpt is essentially sold well below cost now to hook people up. In 5 years from now once no one will be able to maintain their productivity without it, (not at all)OpenAI will raise the prices 100x and everyone will pay begrudgingly. Then they will raise them 100x more and people will pay too... Unless there is a viable alternative. This is why I(and many like me) are working on having my own.