Heh, so it totally depends on the use case. I use GPT4 constantly to chat through issues I am working on and get different perspectives. I cannot do that with

Heh, so it totally depends on the use case. I use GPT4 constantly to chat through issues I am working on and get different perspectives. I cannot do that with local models. On the other hand, I have been processing a ton of text transcripts with a fine tuned llama2 13b model i've been working on, and for the tasks I have fine-tuned on, my local model is producing better results than GPT4, often taking a task that I had to do in multiple steps with GPT4, and being able to complete it in a single shot. I can run my local model through vLLM on my workstation at around the same tokens/sec as I can spend maxing out my API limits with GPT3.5-turbo (~$20/hr) while running on 2x 3090's. I'm hitting the vLLM (OpenAI clone) chat/completions endpoint. My model implements the HF chat_templates feature, and I worked on adding support for that to vLLM: https://github.com/vllm-project/vllm/pull/1756 https://github.com/vllm-project/vllm/pull/1756 (llama.cpp is talking about adding support for it too) so I could easily swap out my model in my data pipeline in place of GPT3.5/GPT4, and I wouldn't have to keep maintaining that code on my side. So, with these transcripts I've been: - Creating YAML knowledge graphs - Summaries - Fixing transcription issues - Creating QA pairs - Identifying sections related to promoting a product or brand - Grading the output that it had generated on accuracy So...it depends on your use case.