Just to double-check, are you using the git versions of transformers and accelerate? I believe the BFloat16 support is pretty recent (22 days ago) and may not b

Just to double-check, are you using the git versions of transformers and accelerate? I believe the BFloat16 support is pretty recent (22 days ago) and may not be in conda/pip: https://github.com/huggingface/accelerate/commit/f13c59f91e20beed3e8017de6e6818666e4e46b7 https://github.com/huggingface/accelerate/commit/f13c59f91e2...