2 ms·

And that is without torch.compile anywhere in the training code. Triton alone should narrow the gap (with torch 2.1, maybe?)