3 ms·
From what I've seen the utilisation is still pretty poor, they're being used so poorly and most companies can get away with less GPUs. Instead of looking at how
by paul-nai 3y ago
From what I've seen the utilisation is still pretty poor, they're being used so poorly and most companies can get away with less GPUs. Instead of looking at how to optimise their workflow they just slap GPUs on
- bushbaba 3y agoI’ve noticed the same. Very low utilization. But they are all used at peak every few weeks. For many companies having more GPU cost to unblock velocity of innovation here is worth it. As the benefits of improvement to your top line revenue far exceeds the GPU cost.
- spott 3y agoAre you talking poor utilization from a "we have a bunch of GPUs sitting idle" sense, or poor utilization from a performance standpoint (can't keep their opus fed with data kinda thing)?
- paul-nai 3y agoKinda both honestly, but more of the fact that code isn’t great for optimising GPUs at run time. For instance no batching, or not realising the CPU is actually the bottleneck