> Unless Nvidia can somehow massively increase memory capacity, it is looking bleak for them in the consumer AI inference space. Who cares about consumer infer

> Unless Nvidia can somehow massively increase memory capacity, it is looking bleak for them in the consumer AI inference space. Who cares about consumer inference? The money is in training because that's where utterly insane amounts of compute capacity are needed, and as long as no competition comes even close to CUDA, NVIDIA has a cash cow to milk. As for potential competition, Apple doesn't sell their silicon to anyone else, and AMD lacks the available manufacture capacity (keep in mind, they also supply the game console market), the driver/tooling quality and most importantly developer trust/ecosystem quality. Everyone is using CUDA.