3 ms·
Not larger vector registers, smaller numbers, so that you can fit more numbers in a vector register without having to make the vector register larger. For CPU-
by Someone 7y ago
Not larger vector registers, smaller numbers, so that you can fit more numbers in a vector register without having to make the vector register larger.
For CPU-bound algorithms, one would expect that bfloat16 in 512 bit vector registers would be about equal in speed to float32 in (hypothetical) 1024 bit vector registers.
Also, for algorithms that are memory-bandwidth bound, halving the size of your numbers will (about) halve memory pressure.
- tomxor 7y ago> Not larger vector registers, smaller numbers, so that you can fit more numbers in a vector register without having to make the vector register larger. Sorry i'm pretty ignorant of AVX so trying to understand... is this because the smallest word size in AVX is 32bit? compared to bfloat16 is using twice the register space? Or rather with the same register space in bfloat16 you can have twice the numbers? (with no negative effects on convergence in NN due to exponent size.)
- Veedrac 7y agoAVX is SIMD, and each AVX register is a large number of bits (256 or 512) that is packed full of a given data type, which can all be operated on at the same time with the same instruction. So if you have a 512 bit register, that could hold either 16 32 bit floating point numbers, or 32 bfloat16s. In the latter case, this means every instruction you do is operating on 32 different values.