3 ms·

I got the impression that anything transformers could be applied to, that would be an advantage. So now I want everything to use them. Doesn't sound like these

• westurner 5y ago
  From https://medium.com/syncedreview/google-replaces-bert-self-attention-with-fourier-transform-92-accuracy-7-times-faster-on-gpus-7a78e3e4ac0e https://medium.com/syncedreview/google-replaces-bert-self-at... : > New research from a Google team proposes replacing the self-attention sublayers with simple linear transformations that “mix” input tokens to significantly speed up the transformer encoder with limited accuracy cost. Even more surprisingly, the team discovers that replacing the self-attention sublayer with a standard, unparameterized Fourier Transform achieves 92 percent of the accuracy of BERT on the GLUE benchmark, with training times that are seven times faster on GPUs and twice as fast on TPUs." Would Transformers (with self-attention) make what things better? Maybe QFT? There are quantum chemical interactions in the brain. Are they necessary or relevant for what fidelity of emulation of a non-discrete brain? https://en.wikipedia.org/wiki/Quantum_Fourier_transform https://en.wikipedia.org/wiki/Quantum_Fourier_transform