3 ms·

Fine tuning these models (at least with PPO or equivalent) requires even more VRAM than inference does, potentially 2-3 times more.

• rusk 5mo ago
  You could use PEFT? Operating on only a subset of weights is fairly standard practice nowadays …
  • andy_ppp 5mo ago
    Yes I used LoRA and it’s fine but I’m not convinced the model doesn’t end up more stupid and less general