5 ms·
Fine-tuning now available for GPT-4o
- hubraumhugo 2y agoCurious to hear some real-world use cases for fine-tuning and what your setup looks like in terms of: - frameworks - hardware - foundation models (OS vs 3rd party) - evaluation & training data Did the performance gains justify the investment for fine-tuning?
- darksaints 2y agoI think vision models have a lot bigger ROI from fine tuning than language models. That being said, I do consider fine-tunes to be helpful in improving smaller models as long as the domain is limited in scope. In other words, fine tuning allows you to get similar performance in smaller models, but improved performance in larger models seems pretty elusive at the moment, albeit somewhat possible with very creatively engineered training datasets. An example of this is DeepSeek-Coder, which can essentially be considered a fine-tune of a fine-tune of a Mixtral model. It performs very similarly to Claude 3.5 Sonnet, which is pretty damn impressive, but it does it at less than 1/10th the cost. What I don't understand though is why anyone would even remotely consider fine tuning a GPT-4o model that they will never fully own, when they could spend the same resources on fine tuning a Llama3.1 model that they will own. And even if you absolutely don't care about ownership (???), why not do a fine tune of an Anthropic model which is already significantly better than GPT-4o. At this point, with the laggard performance of OpenAI and their shameless attempts at regulatory hostility to competitors, I can't imagine ever giving them any of my money, let alone owning my derivative work.
- simonw 2y ago"An example of this is DeepSeek-Coder, which can essentially be considered a fine-tune of a fine-tune of a Mixtral model" I've not heard that anywhere else. My impression from https://huggingface.co/deepseek-ai/deepseek-coder-33b-base https://huggingface.co/deepseek-ai/deepseek-coder-33b-base was that DeepSeek-Coder was trained from scratch.
- darksaints 2y agoThe current deepseek-coder version (v2) is actually a fine tune off of the deepseek v2 model, and was not trained from scratch. I’m now getting conflicting information about the origin of the deepseek MOE framework, so I may be wrong about it starting with a Mixtral model.
- jstummbillig 2y agoWhen Cosine fine tunes a model to be generally better at coding than gpt-4o that leaves me vaguely confused about the role that each part plays in building the best LLMs (and I would not be shocked to learn that this confusion was also SOTA)
- everforward 2y agoI don’t think it’s all that surprising. VRAM is much smaller than all the data the bot was trained on, so some stuff gets dropped or “lossily compressed”. In the case of foundation models, this is likely a bit of everything. Fine tunes take that and change what gets dropped or compressed. Rather than somewhat evenly forgetting things, it forgets very little of the knowledge they’re targeting (like coding) and in exchange drops much more of “everything else”. I don’t believe it works this way, but in a sense, it’s like GPT4o is a 400B parameter model but only devotes 20B parameters to coding because the rest are taken up by Wikipedia and knowing French and what not. A 70B coding fine-tune might be able to devote more than 20B parameters to coding, in exchange for only speaking English, having very little encyclopedic knowledge, etc. It’s kind of like CPUs vs ASICs. CPUs (GPT4o) will perform better on average at random tasks, while ASICs (finetunes) do dramatically better on the task they were built for, even if they have a lower transistor count.
- jstummbillig 2y agoBut is that not what MoE is supposed to solve already? Why no it bake whatever Cosine does into the coding expert?
- everforward 2y agoI am not an expert or even particularly knowledgeable about this, so treat the following as “armchair speculation by an enthusiast”. Wikipedia says that as of 2023, each expert is ~10 billion parameters, so they’re still much smaller than something like Deepseek Coder 70B. I’m not sure why they’re so small though. I don’t know whether there’s some kind of architectural issue or super linear scaling somewhere preventing them from growing, or if they’re just trying to keep inference costs down.
- simonw 2y agoI'm still looking forward to a fine-tuning vendor publishing an interactive real-world example of one of their fine-tuned models, accompanied by the exact training data that was used to create it. OpenAI's examples here are artificial, and there's no mechanism to try out the results: https://platform.openai.com/docs/guides/fine-tuning/fine-tuning-examples https://platform.openai.com/docs/guides/fine-tuning/fine-tun... Fine-tuning is expensive in terms of both time and money (more so time these days, a lot of the vendors have free trials now). Before I put that work in I want to get a much better idea of the kind of results I can expect.
- bob1029 2y agoFine tuning always felt like something that only works at incredible scale with a lot of data. Prompt engineering, RAG, etc. provide so much more uplift per unit time invested. Maybe fine tuning makes sense if you have a 9 figure budget and thousands of humans to throw at it. I don't feel this works in a typical startup ecosystem. I couldn't move the needle with the amount of data I had on hand.
- btown 2y agoTo this point, guaranteeing that your fine-tuning doesn't actually weaken the hallucination-resistance of state-of-the-art base models, by overfitting to specific examples in your training set, is a really hard thing to measure. Unless you have robust ways of validating LLM output so that you can do real cross-validation, few-shot prompts, where you can instruct the LLM exactly how much you want it to generalize from those examples, may be far better at achieving many goals.
- qudat 2y agoWhat about techniques like LoRA?
- bob1029 2y agoI thought these models were already leveraging this technique (or something approximating it) as part of their fine tuning offering.
- satvikpendem 2y agoHow are people feeling with regards to GPT-4o versus Claude 3.5 Sonnet? I recently watched this Primeagen video [0] about how, because LLMs don't actually understand anything (yes, AI Effect included [1]), one does not actually gain as much usefulness as they'd expect, especially with subtly wrong outputs. Over time, it just wastes way more time and becomes a form of learned helplessness (and yes, I do know about Socrates' dialogue, I saw it originally elsewhere on HN and had been quoting it for some time [2]). [0] https://www.youtube.com/watch?v=1-hk3JaGlSU https://www.youtube.com/watch?v=1-hk3JaGlSU [1] https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect [2] https://news.ycombinator.com/item?id=40920318 https://news.ycombinator.com/item?id=40920318
- carbocation 2y agoI feel like this comment was made in good faith so I wanted to explain my downvote: I think it's just too far off-topic for this particular announcement. It's better as a separate discussion.
- satvikpendem 2y agoThanks, however I thought it was fairly aligned simply due to 4o receiving the same sorts of functionality as Claude had for a while.
- logicchains 2y agoI'll also explain my downvote, it's because of the assertion that LLMs "don't actually understand anything", which to anyone who's actually successfully used LLMs to solve a difficult problem is clearly false, unless you use some contrived definition of the word "understand" that doesn't match how the word is actually used in normal conversation.
- teaearlgraycold 2y agoThe bigger ones have gained a rough understanding of a few systems [1]. Which is really impressive and gives an answer to the Chinese Room experiment. In my experience they don’t understand a lot of things I ask about very well. But the fact that they understand anything at all is impressive. 1. https://danangell.com/blog/posts/gpt-understands/ https://danangell.com/blog/posts/gpt-understands/
- tines 2y agoWhat kind of data or data formatting do you need to fine-tune GPT-4o? I'd love to throw a bunch of documentation at it and let it learn, but I don't have the resources to extract knowledge, format it as questions and answers, etc.
- simonw 2y agoFine-tuning generally isn't an effective way to add extra knowledge from things documentation - my understanding is that the vast amounts of knowledge in the original training data tend to overwhelm any extra knowledge you try to add by fine-tuning. OpenAI's documentation has good examples of how the data should be formatted (and when it's appropriate to fine-tune): https://platform.openai.com/docs/guides/fine-tuning/fine-tuning-examples https://platform.openai.com/docs/guides/fine-tuning/fine-tun...
- floam 2y agohttps://platform.openai.com/docs/guides/fine-tuning https://platform.openai.com/docs/guides/fine-tuning
- gamegoblin 2y agoSeems like their competition (Google, Anthropic) have both shipped prompt caching, which is a lot more developer-friendly and gets a lot of the same benefits as fine-tuning. You just cache a prompt with a ton of examples that you would have otherwise fine-tuned on. You can trivially update that prompt whenever, no asynchronous fine-tuning job needed. Wonder if OpenAI will stick with fine-tuning or go towards prompt caching (or both). Fine-tuning has uses, but prompt caching gets you 99% of the benefits for 1% the effort.
- eightysixfour 2y agoThey, in my experience, result in different outcomes. Fine-tuning is better at shaping the response, for example getting a correct JSON format, or making the responses shorters and concise, or use emojis, or whatever. Long prompts are better for in-context learning, fine-tuning doesn't seem to impart knowledge well, it just increases or decreases the likelihoods of existing knowledge coming out.
- maeil 2y agoGenuinely curious as you have experience with fine-tuning, which I don't yet. Given how simple it has become to get the correct response shape, do you feel there's still a point to it? If you need JSON and want to be very sure you can just use function calling, if you only need simple boundaries, use Claude-like XML, if it's something very complicated you can give a few shots. From my understanding, fine-tuning allows for reducing model size, meaning less latency and cost. That seems like it would be the biggest advantage, no?
- eightysixfour 2y ago> Given how simple it has become to get the correct response shape, do you feel there's still a point to it? I am personally working on systems which don't do mrequire uch structured data output; my work is more around structure, style, etc. Fine-tuning is more effective and more consistent for me in those contexts. One thing to remember is that the big models, like 4o and Claude, both are fine-tuned for chat interactions. This tuning can often make them dumber, more verbose, etc. because that's what the human feedback testing likes. You can look at some of the rating tests on chatbot arena as an example where two give the same answer, but people have voted more favorably for the one that delivers it with "more personality." There are SLMs that are better tuned for specific use cases and outperform the big models as a result of this, and the examples OpenAI showed in the article make it clear the big models can benefit from this as well. > From my understanding, fine-tuning allows for reducing model size, meaning less latency and cost. That seems like it would be the biggest advantage, no? This is partially correct, fine-tuning can be a step of this process. The full pipeline looks more like: 1. Create a prompt which gets a large model to output (mostly) correct responses. 2. Build a dataset of those inputs/outputs, probably with human or LLM curation/judging in the loop since some will still be wrong. 3. Fine tune a small model on those inputs/outputs. Now you have a smaller model which behaves more like the large model you were able to prompt engineer into instruction following.
- thomasahle 2y agoDo any vendors allow fine-tuning on images yet?
- Someone1234 2y agoWhen are they going to fix ChatGPT Classic not using Custom Instructions?
- chartpath 2y agoSome much focus on fine-tuning when it can actively make performance on reasoning and planning benchmarks worse (over a baseline of already worse-than-coin-toss). Why not give us nice things for integrating with knowledge graphs and rules engines pretty please?
- xyst 2y ago[flagged]
- simultsop 2y agoJust catching up the phrase of marketing "this is just the start ...". Isn't this overused?
- underyx 2y agoHow do y'all deploy fine-tuned models? I have separate projects for staging and prod, but it doesn't seem like a fine-tune can be shared across projects. Am I wrong to split projects by env? Am I expected to run fine-tunes separately per env (surely not)? Am I missing an option to share fine-tunes across projects?
- simonw 2y agoAre you talking about fine-tuned models that you host yourself, or fine-tuned models from a hosted provider like OpenAI? What do you mean by an "env" here?
- underyx 2y agoI mean OpenAI hosted fine-tunes (same as referenced in OP.) I have a staging deployment and a production deployment. Ideally anything that I roll out to production, I can try on staging first — including switching from gpt-4o to a fine-tuned gpt-4o. I don't want the production API key to be accessible by my staging app, so I have two separate projects within the OpenAI dashboard. One is called my-app-prod, and the other is my-app-staging. To illustrate the problem further, I also have infrastructure to eval models before changing what production is running. The eval infrastructure also has its own OpenAI project, so that I can set separate billing limits on running evals. Any fine-tuned model needs to be benchmarked first, but again, I'm not sure how to make the same fine-tune available to both the eval project, and the production app project.
- john_allard 2y agoHey, engineer on the OpenAI fine-tuning team here. We know this is something of a pain and we’re trying to come up with a way to allow you to share / move models across projects. If you really need to share a model across projects right now the best way to do it is to train the model in the default project, it will then be available to use and all other projects. That’s not an ideal solution obviously but is the only mechanism available currently
- FlamingMoe 2y agoIt's wild how quickly the AI landscape shifts. Just 6 months ago, I couldn't go a day without using ChatGPT. Now, it's like finding an old flip phone in a drawer. I've gone full Claude convert. Wonder where I'll be six months from now.
- aantix 2y agoGithub's Copilot Workspace or Cursor's new composer beta feature. That's where you'll be in 6 months. Multi-file code suggestions with an intimate understanding of the entire code base. https://x.com/aantix/status/1819794837375263228 https://x.com/aantix/status/1819794837375263228 P.S. Or Plandex, https://plandex.ai/ https://plandex.ai/
- rd 2y agoCursor + Sonnet feels like a cheat code right now. Feels similar to the excitement I had the first day I ever used GitHub copilot. I saw this this morning - it really made me think about what the future of 1-man non technical founder startups will look like. https://x.com/0xluffyb/status/1825854097481736479?s=42&t=7-X_YJTpXsyTNSPsjv6tgg https://x.com/0xluffyb/status/1825854097481736479?s=42&t=7-X...
- greenchair 2y agoI was going to try copilot workspace till during signup process it asked for access to my private repos then noped out of there. Why would they need that unless they wanted to use it to train off my data?
- bachback 2y agohow you find it better? haven't seen the benefits
- machiaweliczny 2y agoI have sub to Claude now due to projects/artifacts (caching really) but aider with new gpt4o works well
- ziofill 2y agoI think I’m not understanding what goes into making this available. Fine-tuning is a known technique, why is it available “only” now?
- HarHarVeryFunny 2y agoBecause this isn't you fine-tuning an open weights model and running it yourself - this is fine-tuning via OpenAI's API and them hosting the resulting model (and everybody else's differently fine-tuned ones) and running it for you.
- WhitneyLand 2y agoSay I want to ask a model about a code base 500k tokens in size. The questions would not be to generate code, but rather to help understand it in a way that require the model to globally reason about everything at once. How might performance compare between: 1.) Using a model like Gemini to load it all into context at once 2.) Using one of the various summarization systems/embedding/RAG etc. 3.) Fine tuning the whole code base into Gpt-4o
- spullara 2y ago1 then 2. Never 3.
- WhitneyLand 2y agoInteresting why would you say that? There’s not been much opportunity previously to easily fine tune a Gpt 4 class model. I haven’t seen anything written up on this being tried.
- Liquix 2y agotools like aider [0] maintain a large context with minimal token usage by reading a repo's .git folder. impressive results [0] https://github.com/paul-gauthier/aider https://github.com/paul-gauthier/aider
- trees101 2y agoalso, aider has added experimental support for prompt caching: https://github.com/paul-gauthier/aider/issues/1086#issuecomment-2297256228 https://github.com/paul-gauthier/aider/issues/1086#issuecomm... I really like the control you get with aider over the LLM context. You can /add or /drop source code, markdown notes. You can /clear the chat. /tokens shows you the context and the cost, you can see what each prompt will cost you. I find aider best used in conjunction with a git diff view in VSCode, I run aider with --no-auto-commits and then manually review each time in VSCode. I'm keen to learn any AI coding workflows if anyone has any links. I've benefitted greatly from tips such as using type hints and documentation for the LLM's benefit.
- highfrequency 2y agoHuh? It says: "Cosine's Genie achieves a SOTA score of 43.8% on the new SWE-bench Verified benchmark" with a link to https://www.swebench.com/ https://www.swebench.com/ But the SWE-bench leaderboard (linked to in the post) doesn't show Cosine Genie at all, instead showing Amazon at the top with 38.8% accuracy. If true, seems wild that Genie, a 10 person startup with $2.5m in funding can actually achieve SOTA results over Google, Amazon, Microsoft, Anthropic, OpenAI etc. It's not like the big players are overlooking this problem of using LLMs to automate software engineering. Anyone have more color on this? I see some speculation online that the training data can easily get contaminated with benchmark questions, but not much careful evidence.
- jstummbillig 2y ago"verified" being the (rather confusing) keyword here. https://openai.com/index/introducing-swe-bench-verified/ https://openai.com/index/introducing-swe-bench-verified/
- highfrequency 2y agoYour link doesn't actually point to the leaderboard. This link does https://www.swebench.com/ https://www.swebench.com/ and you can click on the "Verified" tab. I don't see any entry for Genie.
- hansonw 2y agoIt looks like they didn't want to make a public submission in order to avoid disclosing the model internals: https://cosine.sh/blog/genie-technical-report#:~:text=SWE%2DBench%20has,SWE%2DBench%20tasks https://cosine.sh/blog/genie-technical-report#:~:text=SWE%2D....
- JamesBarney 2y agoIt's with an asterisk. Here's their comment from cosine's website. Note SWE-Bench has recently modified their submission requirements, now asking for the full working process of our AI model in addition to the final results -their condition to have us appear on the offical leaderboard. This change poses a significant challenge for us, as our proprietary methodology is evident in these internal processes. Publicly sharing this information would essentially open-source our approach, undermining the competitive advantage we’ve worked hard to develop. For now, we’ve decided to keep our model’s internal workings confidential. However we’ve made the model’s final outputs publicly available on GitHub for independent verification. These outputs clearly demonstrate our model’s 30% success rate on the SWE-Bench tasks.
- Der_Einzige 2y agoThere seems to be no information on how this fine-tuning is implemented, and this matters, because parameter efficient techniques minimize catestrophic forgetting, which is most of what causes fine-tuned models to perform worse. Until this happens, and until we have flexibility of fine-tuning techniques, I will push investors still towards PyReft, Lora, related, and the open source models where all of these techniques work.
- celdon25 2y ago> Fine-tuned models remain entirely under your control Also on the very next paragraph: > We’ve also implemented layered safety mitigations for fine-tuned models to ensure they aren’t being misused. Well done, Sam
- bionhoward 2y agostill has a customer noncompete so you’re still paying to get wrecked long term
- ajeeto 2y ago[dead]
- ajeeto 2y ago[dead]