3 ms·
This is the takeaway here: That's how they have been serving it at scale as Ox-Alpha. This is a definitional moment.- Further quote: "Compared with our initia
by Bluestein 2mo ago
This is the takeaway here: That's how they have been serving it at scale as Ox-Alpha. This is a definitional moment.-
Further quote:
"Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale."
https://z.ai/blog/glm-5.3-flash https://z.ai/blog/glm-5.3-flash
- xtracto 2mo agoAnyone knows what are those Chinese chips? Can they be bought? (Assuming im not i the US, And actually im in a 3rd world country).
- gunalx 2mo agoThey are high end really expensive Huawei ascend GPUs. It is kinda bruteforcing the performance on a older semiconductor processing tech, so total production is pretty low.
- ayewo 2mo agoPerhaps these may be Huawei Ascend chips. https://en.wikipedia.org/wiki/HiSilicon#Ascend_910 https://en.wikipedia.org/wiki/HiSilicon#Ascend_910 https://medium.com/@huaweiclouddevelper/a-brief-introduction-to-huawei-ascend-cloud-cbef8f25bc34 https://medium.com/@huaweiclouddevelper/a-brief-introduction...
- bean469 2mo ago> comparable to mainstream NVIDIA GPUs By this they probably mean RTX series GPUs? If so, then they are not comparing the hardware efficiency with the A100 / H100, etc. that are commonly used for training models