Having spent a fair amount of time over the past few years as someone working on top of both Nvidia and AMD toolchains (and having spent decades working up and

Having spent a fair amount of time over the past few years as someone working on top of both Nvidia and AMD toolchains (and having spent decades working up and down the stack from mcus, mobile device, to webscale stacks), I have an alternative take. I'm smart and experienced enough, and the people working on the toolchains are capable and even smarter, but these systems are just extremely hard/complex. The hardware is buggy, the firmware is buggy, the drivers are buggy, every layer above it (compilers, kernels, multiple layers of frameworks) also all buggy. Add on top of that everyone is implementing as fast as they can, every part is changing constantly, and also a good chunk of the code being used is being written by researchers and grad students. Oh, and the hardware changes dramatically every couple years. Sure all tech is swiss cheese, but this is a particularly unstable/treacherous version of that atm. BTW, shareholders obviously have no idea whether what's being sold works well or not, but there's no financial incentive (in terms of selling stuff/profiting) if your product doesn't even work/work well enough to compete. And if it were so easy, you would see any number of hardware startups take over from the incumbents. Sure there are network effects, but if I could run PyTorch (or really, just train and inference my models easily) faster and cheaper, me, and I assume lots of other people would switch in a heartbeat. The fact that no one (besides Google arguably if you count having to switch all your implementations over for TPUs) has been able to do it, whether they be startups or multi-billion dollar companies (including Amazon, Meta, and Microsoft, all who have direct financial incentive to and have been trying), points to the problem being elsewhere.