3 ms·
Exactly what I was thinking. What sort of latency do you think one would get with 8x B200 Blackwell chips? Do you think 1500 tokens/sec would be achievable in
by cgdl 1y ago
Exactly what I was thinking.
What sort of latency do you think one would get with 8x B200 Blackwell chips? Do you think 1500 tokens/sec would be achievable in that setup?