3 ms·

> The reason to use float16 isn’t to make individual operations faster, its to fit more numbers vector registers. For example, AVX512 vector registers are 512 b

by tomxor 7y ago

> The reason to use float16 isn’t to make individual operations faster, its to fit more numbers vector registers. For example, AVX512 vector registers are 512 bits. That’s 8 64-bit floats, 16 32-bit ones, or 32 16-bit ones. I'm unclear on the advantage you are trying to explain. If both AVX and bfloat are SIMD instructions that cannot be the reason implementing bfloat is better. I'm expecting something like "bfloat16 is more specialised so it can have larger registers" or something? [Edit] Sorry re-reading your comment, I think you are trying to say that. The key part being: > fit more numbers vector registers (vs AVX i assume), so adding bfloat16 would provide more registers vs AVX with similar gate usage due to greater specialization?

Someone 7y ago
Not larger vector registers, smaller numbers, so that you can fit more numbers in a vector register without having to make the vector register larger. For CPU-bound algorithms, one would expect that bfloat16 in 512 bit vector registers would be about equal in speed to float32 in (hypothetical) 1024 bit vector registers. Also, for algorithms that are memory-bandwidth bound, halving the size of your numbers will (about) halve memory pressure.
- tomxor 7y ago
  > Not larger vector registers, smaller numbers, so that you can fit more numbers in a vector register without having to make the vector register larger. Sorry i'm pretty ignorant of AVX so trying to understand... is this because the smallest word size in AVX is 32bit? compared to bfloat16 is using twice the register space? Or rather with the same register space in bfloat16 you can have twice the numbers? (with no negative effects on convergence in NN due to exponent size.)
  - Veedrac 7y ago
    AVX is SIMD, and each AVX register is a large number of bits (256 or 512) that is packed full of a given data type, which can all be operated on at the same time with the same instruction. So if you have a 512 bit register, that could hold either 16 32 bit floating point numbers, or 32 bfloat16s. In the latter case, this means every instruction you do is operating on 32 different values.

by tomxor 7y ago

Someone 7y ago

Not larger vector registers, smaller numbers, so that you can fit more numbers in a vector register without having to make the vector register larger. For CPU-bound algorithms, one would expect that bfloat16 in 512 bit vector registers would be about equal in speed to float32 in (hypothetical) 1024 bit vector registers. Also, for algorithms that are memory-bandwidth bound, halving the size of your numbers will (about) halve memory pressure.

tomxor 7y ago
> Not larger vector registers, smaller numbers, so that you can fit more numbers in a vector register without having to make the vector register larger. Sorry i'm pretty ignorant of AVX so trying to understand... is this because the smallest word size in AVX is 32bit? compared to bfloat16 is using twice the register space? Or rather with the same register space in bfloat16 you can have twice the numbers? (with no negative effects on convergence in NN due to exponent size.)
- Veedrac 7y ago
  AVX is SIMD, and each AVX register is a large number of bits (256 or 512) that is packed full of a given data type, which can all be operated on at the same time with the same instruction. So if you have a 512 bit register, that could hold either 16 32 bit floating point numbers, or 32 bfloat16s. In the latter case, this means every instruction you do is operating on 32 different values.