2 ms·

What is a good fine-tuning script for Mistral and LLaMA3 on an A100?

• strickvl 2y ago
  Depends a bit where you’re running etc. This works for Modal, e.g., but they’re just using axolotl under the hood so you can just connect to whatever cloud provider of choice you’re using and then run axolotl straight. I did my finetunes across local GPUs, but it would have been just as easy to do it in a cloud environment using the same axolotl config.
• swalsh 2y ago
  Unsloth is a great tool, super fast.
  • strickvl 2y ago
    But still only single GPU for now. I also heard great things about it, but wanted to make the maximum use of my multi-GPU local setup.
    • swalsh 2y ago
      Last time I emailed them, I think they said they're trying to release pro in July... so should be coming soon.