12 ms·
Qwen3.5 Fine-Tuning Guide
- clueless 8mo agoWhat are some sample real world cases folks are using to fine tune their own small/medium models?
- danielhanchen 8mo agoOh I wrote up a post on X on this exact question! https://x.com/danielhanchen/status/1979389893165060345?s=20 https://x.com/danielhanchen/status/1979389893165060345?s=20 1. Cursor used online RL to get +28% approval rate: https://cursor.com/blog/tab-rl https://cursor.com/blog/tab-rl 2. Vercel used RFT for their AutoFix model for V0: https://vercel.com/blog/v0-composite-model-family https://vercel.com/blog/v0-composite-model-family 3. Perplexity's Sonar for Deep Research Reasoning I think was a finetuned model: https://docs.perplexity.ai/docs/getting-started/overview https://docs.perplexity.ai/docs/getting-started/overview 4. Doordash uses LoRA, QLoRA for a "Generalized Attribute Extraction model" https://careersatdoordash.com/blog/unleashing-the-power-of-large-language-models-at-doordash-for-a-seamless-shopping-adventure/ https://careersatdoordash.com/blog/unleashing-the-power-of-l... 5. NASA flood water detection https://earthdata.nasa.gov/news/nasa-ibm- https://earthdata.nasa.gov/news/nasa-ibm- openly-release-geospatial-ai-foundation-model-nasa-earth-observation-data6 6. Online RL for robotics - imagine you teaching a robot in the future via some mini finetuning 7. OpenAI's RFT page has more: https://developers.openai.com/api/docs/guides/rft-use-cases https://developers.openai.com/api/docs/guides/rft-use-cases 8. For larger models - https://www.mercor.com/blog/expert-data-drives-model-performance/ https://www.mercor.com/blog/expert-data-drives-model-perform...
- azath92 8mo agoOnly to prompt thought on this exact question, im interested in answers: I just ran a benchmark against haiku of a very simple document classification task that at the moment we farm out to haiku in parallel. very naive same prompt system via same api AWS bedrock, and can see that the a few of the 4b models are pretty good match, and could be easily run locally or just for cheap via a hosted provider. The "how much data and how much improvement" is a question i dont have a good intuition for anymore. I dont even have an order of magnitude guess on those two axis. Heres raw numbers to spark discussion: | Model | DocType% | Year% | Subject% | In $/MTok | |---------------|----------|-------|----------|-----------| | llama-70b -----| 83 | 98 | 96 | $0.72 | | gpt-oss-20b --| 83 | 97 | 92 | $0.07 | | ministral-14b -| 84 | 100 | 90 | $0.20 | | gemma-4b ----| 75 | 93 | 91 | $0.04 | | glm-flash-30b -| 83 | 93 | 90 | $0.07 | | llama-1b ------| 47 | 90 | 58 | $0.10 | percents are doc type (categorical), year, and subject name match against haiku. just uses the first 4 pages. in the old world where these were my own in house models, id be interested in seeing if i could uplift those nubmers with traingin, but i haven't done that with the new LLMs in a while. keen to get even a finger to the air if possible. Can easily generate tens of thousands of examples. Might try myself, but always keen for an opinion. _edit for table formatting_
- airstrike 8mo agoif you add 2 spaces at the start of the line, you turn it into a code block like this
- andai 8mo ago| Model | DocType% | Year% | Subject% | In $/MTok | |----------------|----|-----|----|-------| | llama-70b -----| 83 | 98 | 96 | $0.72 | | gpt-oss-20b ---| 83 | 97 | 92 | $0.07 | | ministral-14b -| 84 | 100 | 90 | $0.20 | | gemma-4b ------| 75 | 93 | 91 | $0.04 | | glm-flash-30b -| 83 | 93 | 90 | $0.07 | | llama-1b ------| 47 | 90 | 58 | $0.10 |
- azath92 8mo agothank you so much! i suffered with this, and now i never will again!
- deleted 8mo ago[deleted]
- deleted 8mo ago[deleted]
- faxmeyourcode 8mo agoLabeling or categorization tasks like this are the bread and butter of small fine tuned models. Especially if you need outputs in a specific json format or whatever. I did an experiment where I did very simple SFT on Mistral 7b and it was extremely good at converting receipt images into structured json outputs and I only used 1,000 examples. The difficulty is trying to get a diverse enough set of examples, evaling, etc. If you have great data with simple input output pairs, you should really give it a shot.
- arkmm 8mo agoYou can fine tune a small LLM with a few thousand examples in just a few hours for a few dollars. It can be a bit tricky to host, but if you share a rough idea of the volume and whether this needs to be real-time or batched, I could list some of the tradeoffs you'd think about. Source: Consulted for a few companies to help them finetune a bunch of LLMs. Typical categorical / data extraction use cases would have ~10x fewer errors at 100x lower inference cost than using the OpenAI models at the time.
- clipclopflop 8mo agoHi! I think this is a pretty good example: https://www.atredis.com/blog/2024/6/3/how-to-train-your-large-language-model https://www.atredis.com/blog/2024/6/3/how-to-train-your-larg...
- ygouzerh 8mo agoI am thinking to fine-tune it to recognize better my handwriting. It already works quite well by default, but my writing is just horrible, so it got trouble sometimes.
- syntaxing 8mo agoAwesome guide, shame how a couple of the Qwen leads got kicked out and replaced with more “business” minded leadership. Hopefully this doesn’t mean the end of the open source era from Qwen.
- danielhanchen 8mo agoOh I think on X a few ago: https://x.com/poezhao0605/status/2029151951167078454 https://x.com/poezhao0605/status/2029151951167078454 - Alibaba's CEO and CTO are having an emergency all hands now! Hope it all goes well!
- antirez 8mo agoFine tuning is a story that is nice to tell but that with modern LLMs makes less and less sense. Modern LLMs are so powerful that they are able to few shot learn complicated things, so a strong prompt and augmenting the generation (given the massive context window of Qwen3.5, too) is usually the best option available. There are models for which fine tuning is great, like image models: there with LoRa you can get good results in many ways. And LLMs of the past, too: it made sense for certain use cases. But now, why? LLMs are already released after seeing (after pre-training) massive amount of datasets for SFT and then RL. Removing the censorship is much more efficiently done with other techniques. So I have a strong feeling that fine tuning will be every day less relevant, and already is quite irrelevant. This, again, in the specific case of LLMs. For other foundational models fine tuning still makes sense and is useful (images, text to speech, ...).
- ranger_danger 8mo agowhere it makes sense IMO is when you need it to know about a large amount of information that's not already in the model, such as a company knowledgebase, code repositories or a trove of specialized legal documents... in that case it's not realistic to try to stuff the context window every time with that information, especially if you're trying to make a responsive chat bot.
- antirez 8mo agoWith the current context windows and the ability those models did RL to work as agents, it's much faster and reliable for them to use tools and find the information before replying. Much better, no hallucinations problems (or a lot less), no fine tuning needed when information changes. I believe it is exactly in this case that fine tuning is no longer useful, and even in the past worked at very different degrees of quality.
- dotancohen 8mo agoWouldn't a RAG make more sense for this use case?
- larodi 8mo ago
- aliljet 8mo agoDoes fine tuning really improve anything above just pure RAG approaches for usee cases that involve tons of direct document context?
- 44za12 8mo agoSpecialised models easily beat SOTA, case in point: https://nehmeailabs.com/flashcheck https://nehmeailabs.com/flashcheck
- octoclaw 8mo ago[dead]
- NitpickLawyer 8mo agoRemember how the tab-next-action model from Cursor was all the rage ~2 years ago when they launched it? That was a fine-tune of a ~70b model (they kinda alluded to this in a podcast).
- krasikra 8mo ago[flagged]
- andai 8mo agoVery interesting. Could you give examples of industrial tasks where lower accuracy is acceptable?
- eitally 8mo agoIndustrial inspection is usually a fairly blunt task and I wouldn't be concerned about accuracy. Especially in high volume environments where training data is plentiful. Think about things like chip placement errors, alignment problems, bad solder joints, missing components.
- deleted 8mo ago[deleted]
- embedding-shape 8mo ago> where latency matters more than raw accuracy – think industrial inspection Huh? Why would industrial inspection, in particular, benefit from lower latency in exchange for accuracy? Sounds a bit backwards, but maybe I'm missing something obvious.
- someotherperson 8mo agoAt a very high level, think fruit sorting[0] where the conveyor belt doesn't stop rolling and you need to rapidly respond, and all the way through to monitoring for things like defects in silicon wafers and root causing it. Some of these issues aren't problematic on their own, but you can aggregate data over time to see if a particular machine, material or process within a factory is degrading over time. This might not be throughout the entire factory but isolated to a particular batch of material or a particular subsection within it. This is not a hypothetical example: this is an active use case. [0] https://www.youtube.com/watch?v=vxff_CnvPek https://www.youtube.com/watch?v=vxff_CnvPek
- bugglebeetle 8mo agoUnfortunately, this looks to only cover the larger MoE models. I imagine the smaller models are what most people would target. 9B just dropped two days ago, so not surprised it’s not explicitly documented, but does use a hybrid mamba architecture that I expect needs some special consideration.
- STARGA 8mo ago[dead]
- deleted 8mo ago[deleted]