1. Test your code with super low batch size. Bad for convergence, good for sanity check before submitting your job to a super computer. Or you can buy a deskto

1. Test your code with super low batch size. Bad for convergence, good for sanity check before submitting your job to a super computer. Or you can buy a desktop machine for the same price as an M1 MacBook with 32GB or 64GB RAM and an RTX2060 or RTX3060 (which support mixed-precision training) and you can actually finetune a reasonable transformer model with a reasonable batch size. E.g., I can finetune a multi-task XLM-RoBERTa base model just fine on an RTX2060, model distillation also works great. Also, there are only so many sanity checks you can do on something as weak (when it comes to neural net training). Sure, you can check if your shapes are correct, loss is actually decreasing, etc. But once you get at the point your model is working, you will have to do dozens of tweaks that you can't reasonably do on an M1 and still want to do locally. tl;dr: why make your life hard with an M1 for deep learning, if you can buy a beefy machine with a reasonable NVIDIA GPU at the same price? Especially if it is for work, your employer should just buy such a machine (and an M1 MacBook for on the go ;)).