5 ms·For single user (batch_size = 1), sure. But that is quite expensive in $/tok.by alecco 2y agoFor single user (batch_size = 1), sure. But that is quite expensive in $/tok.