3 ms·

I think it would be much nicer, although unrealistic at the moment given the number of combinations of GPU vendors and variety of hardware, to directly target t

I think it would be much nicer, although unrealistic at the moment given the number of combinations of GPU vendors and variety of hardware, to directly target the underneath GPU ISA machine code. Since I can write a simple compiler to target x64 machine code, it should be possible to write one to target my GPU. Though, I'm certain that vendor lock is probably more profitable for them.

matthewfcarlson 9d ago
I don’t know for sure but I’m pretty sure the ISA changes quite frequently for Nvidia.
cmrdporcupine 9d ago
My thoughts on this, as a person who has recently coming around to working in this space is that up to now the convenience and "simplicity" of working in CUDA as it is has been a giant moat for NVIDIA. Having a whole toolchain with a C++ dialect and a giant extant pile of code out there that looked familiar to people meant they've "won" the AI wars. And in that context NVIDIA had every motivation to keep their SDK somewhat abstracted higher up the chain and fully under their control and then be free to innovate in the lower bits. And this served them well as well as their customers. My sense is that now with agent driven development this is basically evaporating. Agents are capable of at least prototyping/writing kernels for any hardware and ISA. e.g. OpenAI built their own custom hardware and ISA for it and then set agents loose on it writing kernels and claims great success. At least they're claiming this. And from my own experiences as a n00b entering this space, I can believe it. TLDR I don't think vendor lock on the software side is going to work out for them as a strategy. But luckily for them they continue to have really good hardware and good access to semiconductor fabrication. But just look at HotChips 2026 a couple weeks ago and look at the huge variety of new inference hardware coming down the pipe which looks completely unlike NVIDIA/CUDA.

matthewfcarlson 9d ago

I don’t know for sure but I’m pretty sure the ISA changes quite frequently for Nvidia.

cmrdporcupine 9d ago

My thoughts on this, as a person who has recently coming around to working in this space is that up to now the convenience and "simplicity" of working in CUDA as it is has been a giant moat for NVIDIA. Having a whole toolchain with a C++ dialect and a giant extant pile of code out there that looked familiar to people meant they've "won" the AI wars. And in that context NVIDIA had every motivation to keep their SDK somewhat abstracted higher up the chain and fully under their control and then be free to innovate in the lower bits. And this served them well as well as their customers. My sense is that now with agent driven development this is basically evaporating. Agents are capable of at least prototyping/writing kernels for any hardware and ISA. e.g. OpenAI built their own custom hardware and ISA for it and then set agents loose on it writing kernels and claims great success. At least they're claiming this. And from my own experiences as a n00b entering this space, I can believe it. TLDR I don't think vendor lock on the software side is going to work out for them as a strategy. But luckily for them they continue to have really good hardware and good access to semiconductor fabrication. But just look at HotChips 2026 a couple weeks ago and look at the huge variety of new inference hardware coming down the pipe which looks completely unlike NVIDIA/CUDA.