I think model providers would love to run their models on a single GPU. The latency and throughput of GPU interconnects is orders of magnitudes worse than acces

I think model providers would love to run their models on a single GPU. The latency and throughput of GPU interconnects is orders of magnitudes worse than accessing VRAM. Cutting out the latency would make the models much more efficient to run, they wouldn't have to pay for such expensive networking. If they got to run it on consumer GPUs even better. Consumer GPUs probably cost something like 5-10x less with regards to raw compute than data center ones. New coding optimized models for single GPUs drop all the time. But it's just a really hard problem to make them good and when the large models are still in the barely good enough phase (I wasn't using agents much before Sonnet 4) it's just not realistic to get something useful locally.