I think vision models have a lot bigger ROI from fine tuning than language models. That being said, I do consider fine-tunes to be helpful in improving smaller

I think vision models have a lot bigger ROI from fine tuning than language models. That being said, I do consider fine-tunes to be helpful in improving smaller models as long as the domain is limited in scope. In other words, fine tuning allows you to get similar performance in smaller models, but improved performance in larger models seems pretty elusive at the moment, albeit somewhat possible with very creatively engineered training datasets. An example of this is DeepSeek-Coder, which can essentially be considered a fine-tune of a fine-tune of a Mixtral model. It performs very similarly to Claude 3.5 Sonnet, which is pretty damn impressive, but it does it at less than 1/10th the cost. What I don't understand though is why anyone would even remotely consider fine tuning a GPT-4o model that they will never fully own, when they could spend the same resources on fine tuning a Llama3.1 model that they will own. And even if you absolutely don't care about ownership (???), why not do a fine tune of an Anthropic model which is already significantly better than GPT-4o. At this point, with the laggard performance of OpenAI and their shameless attempts at regulatory hostility to competitors, I can't imagine ever giving them any of my money, let alone owning my derivative work.