4 ms·

If anyone's interested, I made Colab notebooks with free GPUs for both GRPO (the algo DeepSeek used) to train a reasoning model from scratch, and also general f

If anyone's interested, I made Colab notebooks with free GPUs for both GRPO (the algo DeepSeek used) to train a reasoning model from scratch, and also general finetuning, which the Berkeley team employed! GRPO notebook for Llama 3.1 8B: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-GRPO.ipynb https://colab.research.google.com/github/unslothai/notebooks... General finetuning notebook: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-Alpaca.ipynb https://colab.research.google.com/github/unslothai/notebooks... The Berkeley team's 17K dataset: https://huggingface.co/datasets/NovaSky-AI/Sky-T1_data_17k https://huggingface.co/datasets/NovaSky-AI/Sky-T1_data_17k Hugging Face also released a 220K dataset: https://huggingface.co/datasets/open-r1/OpenR1-Math-220k https://huggingface.co/datasets/open-r1/OpenR1-Math-220k

threecheese 2y ago
How long does this take on a free tier T4? This is really neat, I’d assumed this type of “playing with the guts” work was more difficult to access as a normie programmer. Looks like something I’d like to try!
- danielhanchen 2y ago
  For GRPO - we also made it much faster, but you might need to wait 2 to 4 hours in the minimum for anything meaningful :) Also you can install Unsloth on your local machine :) Kaggle has 2x Tesla T4s as well for free for 30 hours per week!