3 ms·
Ask HN: How to Fine Tune LLMs?
I wanted to play around with fine tuning LLMs on my Mac. Any recommendations on how (or) where to start? Is there a blog you'd suggest?
- PaulHoule 3y agoThese instructions got me started https://huggingface.co/docs/transformers/training https://huggingface.co/docs/transformers/training with the caveat that the default training rate is too high and you get better results with 2e-5
- krishadi 3y agoGreat! Thanks...
- verdverm 3y agoThere are a number of ways, with increasing methods and capabilities. It's going to depend on what you are trying to do and the constraints you have Google put out a paper last week that talks about some of this and proposed a new method that outperforms previous: https://news.ycombinator.com/item?id=35810663 https://news.ycombinator.com/item?id=35810663