3 ms·
With eval rate numbers: - phi4: 12 tokens/s - mistral-small: 9 tokens/s On Nvidia RTX 4090 laptop: - phi4: 36 tokens/s - mistral-small: 16 tokens/s
by mike31fr 2y ago
With eval rate numbers:
- phi4: 12 tokens/s
- mistral-small: 9 tokens/s
On Nvidia RTX 4090 laptop:
- phi4: 36 tokens/s
- mistral-small: 16 tokens/s