Fine tuning is training. Just done on smaller and higher quality dataset. For pretraining, RAM is the least of the issue. Even for smallest llama 2 they used 1

Fine tuning is training. Just done on smaller and higher quality dataset. For pretraining, RAM is the least of the issue. Even for smallest llama 2 they used 12 years of A100 GPU time.