23 ms·
17k token/sec is $0.18/chip/hr for the size of H100 chip if they want to compete with the market rate[1]. But 17k token/sec could lead to some new usecases. [1
by YetAnotherNick 8mo ago
17k token/sec is $0.18/chip/hr for the size of H100 chip if they want to compete with the market rate[1]. But 17k token/sec could lead to some new usecases.
[1]: https://artificialanalysis.ai/models/llama-3-1-instruct-8b/providers?end-to-end-response-time=end-to-end-response-time-vs-price https://artificialanalysis.ai/models/llama-3-1-instruct-8b/p...