3 ms·How fast are these small models on a 4090, is it like 100ms ? 500ms ?by simlevesque 3y agoHow fast are these small models on a 4090, is it like 100ms ? 500ms ?pulse7 3y agoMistral-7B gives you 80 tokens/second on 4090. So this one will be faster...emadm 3y agoIt’s about twice the speed