bfloat16 mostly matters for training stability, not for inference. M1 is inefficient for training any reasonably sized model and most inference efforts target 4

bfloat16 mostly matters for training stability, not for inference. M1 is inefficient for training any reasonably sized model and most inference efforts target 4-8 bits, even.