Hey! I managed to make QLoRA finetuning of Gemma 7b 2.43x faster than HF + Flash Attention 2 with 57.5% less VRAM use. When compared to vanilla HF, Unsloth is 2

Hey! I managed to make QLoRA finetuning of Gemma 7b 2.43x faster than HF + Flash Attention 2 with 57.5% less VRAM use. When compared to vanilla HF, Unsloth is 2.53x faster and uses 70% less VRAM. On 1x A100 80GB GPU, Unsloth can fit 40K total tokens (8192 * bsz of 5), whilst FA2 can fit ~15K tokens and vanilla HF can fit 9K tokens. I also found a RoPE bug, and posted my learnings at https://unsloth.ai/blog/gemma https://unsloth.ai/blog/gemma