3 ms·

I’m betting the opposite: new model architectures will unlock greater abilities at the cost of massive compute.

• mistercow 2y ago
  Even so, massive compute doesn’t necessarily mean GPU-friendly compute. We could see a breakthrough in analog or neuromorphic hardware, for example, where Nvidia isn’t well positioned. Or we could see a training breakthrough which is far more efficient, but bottlenecked on single core performance, or just branch-heavy performance. You can imagine scenarios like that where GPUs still play a role, but where even today’s top of the line GPUs are way over the top compared to the CPU bottleneck. If one of those scenarios happens, maybe Nvidia can pivot, or if we see analog take over, we could see something really bizarre like a dark horse like Seagate taking over by pivoting from SSDs, just because their manufacturing pipeline is more compatible.