3 ms·

Yes but then the batch size is in 100s or even 1000s. These GPU doesn't serve just 1 user at a time.