4 ms·

symbol manipulation in transformers is fp arithmetic?

• wizzwizz4 8mo ago
  That's an implementation detail. The behaviour of trained transformer models remains similar even if you quantise them to 4-bit floats, or make every floating point operation noisy. This model only works if you use double-precision floating point.