4 ms·
It's the number of tokens the model can output in one pass. There's subtle differences between running it multiple times to get a bigger output and running it o
by xena 2y ago
It's the number of tokens the model can output in one pass. There's subtle differences between running it multiple times to get a bigger output and running it once to get a bigger output. These are things that only really show up when you integrate these models into production code.