3 ms·
Can someone share a good tutorial how to prepare the data? And for fine tuning, does a 3090 have enough VRAM? I want to do what the author mentioned by fine tun
by syntaxing 3y ago
Can someone share a good tutorial how to prepare the data? And for fine tuning, does a 3090 have enough VRAM? I want to do what the author mentioned by fine tuning the model on my personal data but I’m not sure how to prepare the data. I tried using vector search + LLM but I find the results very subpar when using a local LLM.
- samlhuillier 3y agoWorking on this now!
- syntaxing 3y agoI’m looking forward to this! Are you using an adapter (I don’t see it mentioned in your article)? I was under the impression you cannot fit 7B at 4 bit since it’ll take 25GB of VRAM or so.
- samlhuillier 3y agoYes using the qlora adapter that hugging face provides with peft
- syntaxing 3y agoahh, I was on my phone before so I must of glimpsed over it, I see it on the last section. Thanks!
- jawerty 3y agoI just streamed this last night https://m.youtube.com/watch?v=TYgtG2Th6fI&t=3998s https://m.youtube.com/watch?v=TYgtG2Th6fI&t=3998s I’ve been live streaming myself fine tuning llama on my GitHub data (to code like me)
- jeremycarter 3y agoFantastic job! Very easy to follow
- jawerty 3y agoThank you! I have some other streams where I do little projects like these check them out
- notpublic 3y agoAs mentioned in the OP's blog post, checkout https://github.com/facebookresearch/llama-recipes.git https://github.com/facebookresearch/llama-recipes.git. specifically files in ft_datasets directory. I am able to finetune meta-llama/Llama-2-13b-chat-hf on a 3090 using instructions from quickstart.ipynb.
- syntaxing 3y agoOh interesting, I didn’t know the documentation expanded so much in this past month.