3 ms·

Have a link? I haven't seen any finetuning scripts in the wild that train a PEFT model on a multrigpu setup yet and would love to play around with one.

• gorbypark 4y ago
  The original Alpaca repo has the training script. The readme has the torchrun command and arguments used for train.py. https://github.com/tatsu-lab/stanford_alpaca/blob/main/train.py https://github.com/tatsu-lab/stanford_alpaca/blob/main/train...
  • nico 4y ago
    Awesome, thank you!