4 ms·

Somehow missed this from WWDC23, but it looks like Sonoma will add support for bfloat16 with Metal, and there's an active PR to add support with the PyTorch MPS

by eoskx 3y ago

Somehow missed this from WWDC23, but it looks like Sonoma will add support for bfloat16 with Metal, and there's an active PR to add support with the PyTorch MPS back-end (PR #99272). Since M2 added bfloat16 support at the hardware level, I'm assuming this will only be supported on M2 Macs. That maxed out Mac Studio M2 w/ 192GB of memory now looks more appealing...

kzrdude 3y ago
Visible in the unofficial documentation for AMX instructions too - M2 only bf16 functionality - https://github.com/corsix/amx/blob/main/matfp.md https://github.com/corsix/amx/blob/main/matfp.md This matfp instruction computes an outer product and is a kernel for matrix multiplication.
- dlewis1788 3y ago
  I didn't even know about Apple's AMX instructions until I clicked on your link. Very interesting - thanks!
my123 3y ago
bf16 in Metal on macOS 14 is supported on all Macs. Emulated in software transparently.
- LoganDark 3y ago
  Yeah, Metal is pretty great because it runs the same on all Macs. Apple is really really good at this.