3 ms·
To my understanding, with batched inference and other "optimizations" you wouldn't get the exact same token predictions even with temp=0.0.
by mh- 5d ago
To my understanding, with batched inference and other "optimizations" you wouldn't get the exact same token predictions even with temp=0.0.