4 ms·

I didn't know that, I might look at the NanoGPT code and torch.dropout docs a bit closer then. Thanks!