Sure, but for inference it’s much easier to work in lower precision. Training with very low precision is also possible, but is often more tricky due to numerica

Sure, but for inference it’s much easier to work in lower precision. Training with very low precision is also possible, but is often more tricky due to numerical stability. A lot of work is going on in 8-bit inference and even 4 bit inference. So, models that need 64 GB in FP32 can do with 16GB VRAM in FP8 or INT8, which is well within the realm of consumer NVIDIA cards. And the latest NVIDIA tensor cores will absolutely destroy Apple Silicon GPUs or the Neural Engine in 8 bit. So, I don’t think it’s really a strong argument. And as someone who is a Mac user and a ML practitioner, I’d be very happy if they started supporting eGPUs again. Apple Silicon has many strengths and the GPU core are fine for many ends, from games to graphics apps. But let’s not pretend that Apple is beating NVIDIA at their own game (yet). That day might come, but currently it only leads to disappointed users in ML forums who were hyped into thinking that their vanilla M2 MacBook Airs can almost compete with a 4090 in training a deep transformer model. (Yes, that happens.)