3 ms·
China has cheap electricity.
by jojobas 7mo ago
China has cheap electricity.
- ericd 7mo agoWell, also, LLM servers get much more efficient with request queue depth >1 - tokens per second per gpu are massively higher with 100 concurrents than 1 on eg vllm.
- DeathArrow 7mo agoYes, but the hardware they use for inference like Huawei Ascend 910C is less efficient than Nvidia H100 used in US due to the difference in the process node.