For those not familiar with Bert transformer arch. You can read a bunch of their torch benchmark code to measure speed up in just the FFF: https://github.com/pb

For those not familiar with Bert transformer arch. You can read a bunch of their torch benchmark code to measure speed up in just the FFF: https://github.com/pbelcak/UltraFastBERT/blob/main/benchmark_pytorch/main.py https://github.com/pbelcak/UltraFastBERT/blob/main/benchmark... some of which is from the pytorch docs here: https://pytorch.org/tutorials/intermediate/torch_compile_tutorial.html https://pytorch.org/tutorials/intermediate/torch_compile_tut..., e.g. the `timed` function and how they generate data. Also its not just the same 12 neurons, its the 12 neurons based on the previous dot product. So some kind of JIT is needed to load the right ones?