3 ms·

I imagine that Nvidia is trying to build a more sustainable moat. If the only things they have that their competitors don’t are a nice development framework, ni

by amluto 3y ago

I imagine that Nvidia is trying to build a more sustainable moat. If the only things they have that their competitors don’t are a nice development framework, nice libraries and nice drivers, it’s not that hard for a customer to get their software working on a competing hardware platform and cut out a bunch of Nvidia’s enormous markup. But if Nvidia also strong buy-in with datacenter operators and an entire platform to magically run people’s applications without having them need to think about how they’re deployed, then they can try for an AWS-like moat in which customers want to avoid the ongoing cost of DIYing their stack.

gitfan86 3y ago
Assuming that the hardware isn't a moat because other people make similar hardware is a mistake. The networking alone is a huge bottleneck at scale. A competitor has to be better at networking AND chips to be competitive.
shiftpgdn 3y ago
Tiny corp/tinygrad has been working for a year+ and has raised something like $5 million dollars to try to get AMD chips up to speed to Nvidia, without much success. Check out twitter where George Hotz has been very vocal about AMD needing to open source their chips to allow someone to help them get up to speed.
- fotcorn 3y ago
  Consumer AMD chips to be precise. AMD is mostly focusing on their datacenter chips (MI300X), and I assume the support for AI workloads there is much better. They might even see their consumer chip undermining sales of datacenter chips. NVIDIA got big because CUDA works on the most crappy notebook GPUs up to their most powerful chips, and AMD should do the same, but focusing their limited number of driver devs on the expensive enterprise hardware makes sense IMHO.
- amluto 3y ago
  I’m a bit surprised AMD hasn’t purchased tinygrad. Or funded or supported them. Or really made any serious move here.
  - paulmd 3y ago
    I think moving geohotz closer to the levers of power and into places where he can see even more sensitive information to squawk on twitter/github probably is the exact opposite of how AMD wants to handle the situation. AI/ML is a rapidly moving field etc and you know geohot is gonna leak it all on twitter as soon as there's anything to announce, which makes it far more difficult for them to pivot later, etc.
- robot 3y ago
  Yes, he was first like, we're totally doing this on AMD! A few weeks later he's like wtf@#! this is the buggiest thing ever
robot 3y ago
"it’s not that hard for a customer to get their software working on a competing hardware platform and cut out a bunch of Nvidia’s enormous markup" Agree and yet none of the contenders were able to work out their software play (Intel, AMD, chip startups) for more than a year which shows how corporates move slow. Google is not selling their TPUs AFAIK and their tooling is completely focused on internal use. So really interesting to see no one else is properly addressing the need even though they have chips (and the chip itself is much simpler than a cpu, a systolic matrix multiplier array).
- amluto 3y ago
  I’m not taking about the chip suppliers moving fast. I’m talking about the users. For example, here’s Stability pulling it off: https://stability.ai/news/putting-the-ai-supercomputer-to-work https://stability.ai/news/putting-the-ai-supercomputer-to-wo... I’m sure this was a lot of work, and Intel surely helped a lot, and there are probably plenty of kludges involved. But it worked, and there’s a lot of money on the table to do things like this.

by amluto 3y ago

gitfan86 3y ago

Assuming that the hardware isn't a moat because other people make similar hardware is a mistake. The networking alone is a huge bottleneck at scale. A competitor has to be better at networking AND chips to be competitive.

shiftpgdn 3y ago

Tiny corp/tinygrad has been working for a year+ and has raised something like $5 million dollars to try to get AMD chips up to speed to Nvidia, without much success. Check out twitter where George Hotz has been very vocal about AMD needing to open source their chips to allow someone to help them get up to speed.

fotcorn 3y ago
Consumer AMD chips to be precise. AMD is mostly focusing on their datacenter chips (MI300X), and I assume the support for AI workloads there is much better. They might even see their consumer chip undermining sales of datacenter chips. NVIDIA got big because CUDA works on the most crappy notebook GPUs up to their most powerful chips, and AMD should do the same, but focusing their limited number of driver devs on the expensive enterprise hardware makes sense IMHO.
amluto 3y ago
I’m a bit surprised AMD hasn’t purchased tinygrad. Or funded or supported them. Or really made any serious move here.
- paulmd 3y ago
  I think moving geohotz closer to the levers of power and into places where he can see even more sensitive information to squawk on twitter/github probably is the exact opposite of how AMD wants to handle the situation. AI/ML is a rapidly moving field etc and you know geohot is gonna leak it all on twitter as soon as there's anything to announce, which makes it far more difficult for them to pivot later, etc.
robot 3y ago
Yes, he was first like, we're totally doing this on AMD! A few weeks later he's like wtf@#! this is the buggiest thing ever

robot 3y ago

"it’s not that hard for a customer to get their software working on a competing hardware platform and cut out a bunch of Nvidia’s enormous markup" Agree and yet none of the contenders were able to work out their software play (Intel, AMD, chip startups) for more than a year which shows how corporates move slow. Google is not selling their TPUs AFAIK and their tooling is completely focused on internal use. So really interesting to see no one else is properly addressing the need even though they have chips (and the chip itself is much simpler than a cpu, a systolic matrix multiplier array).

amluto 3y ago
I’m not taking about the chip suppliers moving fast. I’m talking about the users. For example, here’s Stability pulling it off: https://stability.ai/news/putting-the-ai-supercomputer-to-work https://stability.ai/news/putting-the-ai-supercomputer-to-wo... I’m sure this was a lot of work, and Intel surely helped a lot, and there are probably plenty of kludges involved. But it worked, and there’s a lot of money on the table to do things like this.