4 ms·
Some you may have seen this but I have a Llama 2 finetuning live coding stream from 2 days ago where I walk through some fundamentals (like RLHF and Lora) and h
by jawerty 3y ago
Some you may have seen this but I have a Llama 2 finetuning live coding stream from 2 days ago where I walk through some fundamentals (like RLHF and Lora) and how to fine-tune LLama 2 using PEFT/Lora on a Google Colab A100 GPU.
In the end with quantization and parameter efficient fine-tuning it only took up 13gb on a single GPU.
Check it out here if you're interested: https://www.youtube.com/watch?v=TYgtG2Th6fI https://www.youtube.com/watch?v=TYgtG2Th6fI
- aledalgrande 3y agoSick I've put it in my queue
- nextworddev 3y agowhich version of Llama 2?
- jawerty 3y agoThe 7b parameter one so not one of the larger ones
- 3abiton 3y agoWhat's special about Llama2?
- Generalia 3y ago[dead]
- soultrees 3y agoCan you do more videos on preparing the data and scraping data? I noticed you seem to be proficient at it by your terminology and it’s something I want to dive into more.
- jawerty 3y agoI have a solid stream about scraping Twitter I did yesterday you should check it out https://www.youtube.com/watch?v=-Hfx9tCeShA&t=5516s https://www.youtube.com/watch?v=-Hfx9tCeShA&t=5516s