5 ms·
Mistral Fine-Tune
- DeathArrow 2y agoCan this be optimized so training of larger variants is possible with a pair of 3090 or 4090?
- qayxc 2y agoThis would involve a lot of effort, but seems to be doable. Here's a starting point that discusses some options: https://huggingface.co/blog/trl-peft https://huggingface.co/blog/trl-peft
- hubraumhugo 2y agoIs finetuning still worth it as models advance so fast? Any real-world use cases? For example, Bloomberg trained a GPT-3.5 class LLM on their financial data last year and soon after GPT-4-8k outperformed it on nearly all finance tasks. We ended up focusing on having high-quality eval data and an architecture that makes switching to new models easy.
- keeptrying 2y agoI think their training methodology was suspect. Also I think there’s a bunch of important techniques that just are t shared.
- barrell 2y agoWhen it comes to traditional NLP related tasks, LLMs are far below dedicated NLP pipelines like POS tagging and feature tagging. However, fine tuning bridges the gap quite a bit between the two. It's a narrow domain, but so is most of programming. I think if you're just training a general purpose LLM to be more inclined towards your data -- no, fine tuning is probably not very relevant. But if you're trying to solve a very specific yet fuzzy problem, and LLMs can get you _part_ of the way there, fine tuning is likely your best bet.
- NhanH 2y agoCan you share a bit more on which tasks are you discussing about?
- anon373839 2y agoNot the OP, but you can take a look at https://spacy.io/usage/spacy-101 https://spacy.io/usage/spacy-101 to get a sense of what traditional NLP tasks look like. These things can be done much faster than LLMs with appropriate tooling (such as spaCy) and they don’t risk hallucination.
- threeseed 2y agoAlso you can do it at scale with SparkNLP: https://sparknlp.org https://sparknlp.org Many of the use cases I've seen for LLMs would actually be better with NLP.
- meandmycode 2y agoWhich kinds of use cases?
- maaaaattttt 2y agoI’ve tried PII anonymization with standard NLP approaches and LLMs have been (way) better at this task in my experience.
- barrell 2y agoIn this comment I was referring to POS tagging and feature extraction. Another use case for fine tuning we have as well is to reduce a 5-shot prompt that we have to run hundreds of times per request down to a “0-shot” (heavy emphasis on the double quotes. Run five shot on gpt-4o a couple thousand times, then fine tune on cohere’s command-r or haiku or llama3 8b or whichever small but mighty llm. You can reduce costs by 99%, or somewhere in that ballpark, without really sacrificing quality on 99% of the queries.
- subroutine 2y agoFunction calling might be one reason. If your app has a lot of custom functions for interacting with tools, fine tuning may be preferred over using context tokens.
- AlexeyBrin 2y agoCan you recommend a tutorial/document about fine tuning a small model for selecting the correct functions ? Would be great to have something local that can replace OpenAI functions/tools API.
- subroutine 2y agoThe linked GitHub article above describes syntax for fine tuning Mistral for function calling. Here is an example of data prepared for fine tuning Llama for function calling... https://huggingface.co/datasets/mzbac/function-calling-llama-3-format-v1.1/viewer https://huggingface.co/datasets/mzbac/function-calling-llama... I'm unaware of any comprehensive guides - we're still in the wild west.
- BaculumMeumEst 2y agoI use a programming language that isn’t publicly available and it would be useful to have a model understand it. AI isn’t something I’m super knowledgeable about but fine-tuning sounds like a good way to accomplish that.
- sp332 2y agoFine tuning can be useful if you need to generate lots of output in a particular format. You can fine-tune on formatted messages, and then the model will generate that automatically. That could save a bunch of tokens explaining the output format in every prompt.
- NeutralForest 2y agoYou can use structured generation instead of fiddling with the prompt, which is unreliable. https://github.com/outlines-dev/outlines https://github.com/outlines-dev/outlines
- codetrotter 2y agoDoes this Python package control the LLMs using something other than text? Or is the end result still that that Python package wraps your prompt with additional text containing additional instructions that become part of the prompt itself?
- tikhonj 2y agoLooks like it actually changes how you do token generation to conform to a given context-free grammar. It's a way to structure how you sample from the model rather than a tweak to the prompt, so it's more efficient and guarantees that the output matches the formal grammar. There's a reference to the paper that describes the method at the bottom of the README: https://arxiv.org/pdf/2307.09702 https://arxiv.org/pdf/2307.09702
- sp332 2y agoThe output of the LLM is not just one token, but a statistical distribution across all possible output tokens. The tool you use to generate output will sample from this distribution with various techniques, and you can put constraints on it like not being too repetitive. Some of them support getting very specific about the allowed output format, e.g. https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md https://github.com/ggerganov/llama.cpp/blob/master/grammars/... So even if the LLM says that an invalid token is the most likely next token, the tool will never select it for output. It will only sample from valid tokens.
- Tenoke 2y agoQuality suffers at very large contexts, and you might want to use your large context for something else (e.g. a long current conversation or a lot of recent data).
- fulafel 2y agoFinance is a very mainstream, broad, english-language dominated field. Most people don't work in a field that is so generalist-LLM friendly.
- kgeist 2y ago>Is finetuning still worth it as models advance so fast? Any real-world use cases? Internal corporate data GPT4 was never exposed to?
- simonw 2y agoI still haven't seen really convincing evidence that fine tuning is useful for the internal corporate data use-case (as opposed to RAG which seems to work really well.)
- ankit219 2y agoFinetuning would never add any new knowledge. For internal corporate data, use RAG or train a model from scratch. Finetuning would not help anyone answer a question from that data.
- Exuma 2y agoHow does one do RAG? I see it mentioned like 20 times
- meiraleal 2y agoyou supply to the LLM the subset of data that you need for a specific prompt. That's RAG.
- behnamoh 2y agoThis article reviews some of the most advanced RAG methods: https://medium.com/@krtarunsingh/advanced-rag-techniques-unlocking-the-next-level-040c205b95bc https://medium.com/@krtarunsingh/advanced-rag-techniques-unl... (not mine)
- qeternity 2y agoThis isn’t true at all. Fine tune with a single sample and see what happens to your model.
- ankit219 2y ago
- probably_wrong 2y agoYes. I have non-English human data annotated in a format that was designed for a very specific health-related study. LLMs haver never seen these annotations, non-English LLMs are not a top priority for companies, and we can only use offline-first ones for data privacy reasons. In this scenario fine-tuning a general purpose LM works wonders.
- newswasboring 2y agoWhat is your strategy for this? Do you finetune when a new flagship model is made available? You said local first, so I'm guessing you might have finetuned llama. But there are llama fine-tunes available which have better performance than the base model. How do you choose?
- probably_wrong 2y agoOur strategy is to take a well-known, battle-tested model as a base, train it, and then hopefully one day release the fine-tuned model on HuggingFace. Other than that, fine-tunes don't really matter for us because not many people are rushing to beat the top models on (say) Georgian POS tagging or Urdu sentiment analysis. As long as the model can turn language into a reasonable vector, we're happy with it.
- CuriouslyC 2y agoFine tuning is generally not worth it over RAG with newer models unless you have a niche data set that's very different from the pretraining data. LoRAs/control vectors are also generally better as they don't induce forgetting or hallucinations, and they're usually "good enough" with RAG.
- hectormalot 2y agoMy sense is that finetuning might still have a role if your use is high-volume and with a narrow/specific goal. For example, we have GPT based summaries of our customer contact calls. It's fairly high volume (a typical bank will handle millions of calls a year). We're considering finetuning for 2 reasons: (1) the current system prompt with instructions is getting quite large* and we see more challenges for GPT to stick to the instructions. We could finetune with our historic summaries and a simplified prompt to slightly improve the performance of the summaries / lower token count (performance). The idea would be to then continue improving our system prompt again from that new starting point. Exploration still to be done though. (2) we might be able to finetune a much smaller model to do what the larger model is currently doing (cost, sustainability) * The larger instruction prompt is because there are lots of specific needs for the summaries (e.g. how to write the summary, what to exclude (health, etc.), what to include (actions taken), and we e.g. give examples of good vs bad summaries to improve performance.
- irthomasthomas 2y agoI think the bitter lesson kicks in shortly after you complete fine tuning, and openai release their next model which performs better at the same task.
- stuckinhell 2y agoNo fine tuning seems pointless right now. you can't even fine tune the frontier models.
- Delmololo 2y agoMy startup has data the Internet doesn't have. I want it to be fixing knowledge based on my data and fine-tune it to be adjusted to my use case already. I also want a.feedbackloop and can't start to send more and more context with the payload just because my feedback loop adds / finetunes data
- ewalk153 2y agoThis seems to be the narrative Microsoft was pushing at the Build conference this week.
- behnamoh 2y agoWeird, because they make one of the best small language models (phi series) which is great for finetuning.
- uyzstvqs 2y agoI'd summarize it as prompting for input, finetuning for output. RAG is a far better option for making a model work with your data/information. Finetuning is better for making it output in a language, style, data format, programming language, etc.
- rldjbpin 2y agonot all domains will receive enough representation in training datasets, due to interest and/or access. in the current landscape, specialized, smaller systems will still be the most efficient way ahead in the near future.
- joaquincabezas 2y agoNow that many open source LLMs are already “useful” it is key to ease the development _around_ them, in particular in a way where users-developers can leverage on “private data” (really, data that is not present in the pre-training of these models). Even if in the repo it is stated that it is optimized for big models (needing A100/H100), I still feel this can help with smaller models more than large ones. We can extend “If you build it, they will come” with “If you provide the tools, the will build”
- jsemrau 2y ago> If you provide the tools, the will build Only makes sense if the incentive of learning that tech provides expectations of future benefits.
- joaquincabezas 2y agothere is also the joy of experimenting a “new and revolutionary” technology. Although yes, with big tech and big salaries there is an economic incentive
- sciencesama 2y agoHow can i train my Whats app chat model ??
- qeternity 2y agoWhat does this mean? How can you train a model on your own WhatsApp messages? To what end? To write like you? For RAG Q&A?
- skerit 2y agoThe "weights" bit is interesting. HuggingFace's SFTTrainer allows you to train on completions only if you want, but while that makes sense for us humans, LLMs are generally better of learning to predict the entire input. This allows you to get the best of both worlds.
- deleted 2y ago[deleted]
- anshumankmr 2y ago>ps, but for smaller models, such as the 7B a single GPU suffices If I wanna do this, what GPU would I need?? I have a 3060 TI (laptop one) and i9 with 16 gigs of ram. I don't have AWS quota, or quota in GCP either. have heard of paperspace, but I want to quickly get started with fine tuning Mistral, cause we are planning to use some of their models for an engagement I am working on.
- qayxc 2y agoCheck out this site: https://www.hardware-corner.net/llm-database/Mistral/ https://www.hardware-corner.net/llm-database/Mistral/ They list hardware requirements by model and you can select your VRAM and system memory to filter available models.
- elpocko 2y agoUnless I'm missing something - this is for inference only, using quantized models. Fine-tuning usually has much higher requirements.
- semi-extrinsic 2y agoUnless you have zero budget, I would strongly suggest going up to a "gaming" desktop computer. A gaming GPU can put out 300 W of heat no problem. If your laptop GPU did the same it would melt, it's going to be limited to around 100 W. And heat dissipation is directly proportionaly to speed. Plus in a desktop, you have the ability to upgrade to faster GPU or even go for multiple GPUs. Caveat: such machines, especially multi-GPU, are loud and produce enough heat to warm up a whole room quickly. And cloud will likely be cheaper if you believe you will not spend more than 10% of the next couple of years running the GPU at full blast.
- washadjeffmad 2y ago>heat dissipation is directly proportionaly to speed. Power limiting Nvidia GPUs up to around 60% of max TDP can offer a substantial reduction to heat and noise with minimal performance loss, but agreed on everything else.
- Hugsun 2y agoIt will be very interesting to see which tools become the de-facto for each of the common LLM use cases. The landscape is so fractured, I feel like I haven't even heard of most tools. I encountered Microsoft's olive the other day and it was completely new to me.
- deleted 2y ago[deleted]