5 ms·
Key paragraph: > However, a small handful of models such as Alibaba’s Qwen and DeepSeek are most popular for inference, with most other models only sporadicall
by djoldman 1y ago
Key paragraph:
> However, a small handful of models such as Alibaba’s Qwen and DeepSeek are most popular for inference, with most other models only sporadically called upon. This leads to resource inefficiency, with 17.7 per cent of GPUs allocated to serve only 1.35 per cent of requests in Alibaba Cloud’s marketplace, the researchers found.
- make3 1y agothese other models are likely much smaller