5 ms·
Finetuning LLMs on a Single GPU Using Gradient Accumulation
- eachro 4y agoI thought Raschka was a tenured professor. Did he leave academia?
- warkdarrior 4y agoFrom https://sebastianraschka.com/ https://sebastianraschka.com/ "I used to hold a position as an Assistant Professor of Statistics at the University of Wisconsin-Madison (on a tenure track from 2018-2025). However, with a heavy heart, I recently resigned in 2023 to concentrate fully on my work at the Lightning AI startup, which I had joined in January 2022."
- hogu 4y agoThis won’t help if your model won’t fit on a single gpu right? So I’m this example your model has to be under 16gb if you memory?