4 ms·

> BUT - is there anything preventing Intel simply adding a ton of RAM to their cards? The market isn't demanding it. For gaming customers, modern games don't

> BUT - is there anything preventing Intel simply adding a ton of RAM to their cards? The market isn't demanding it. For gaming customers, modern games don't need more VRAM, 2k-4k textures can fit into 12-16GB of VRAM and 8k textures mostly don't improve the look at a 4k monitor resolution, unless these textures are stretched over large objects. With upscaling technology, Nvidia customers aren't running at 4k native resolution, so 2k textures do fine for 1080p upscaled to 1440p or 1440p upscaled to 4k. Gaming customers want raster performance primarily, assuming VRAM is above 10-12GB. We will see retail demand for more VRAM when monitor resolutions exceed 4k, and this won't happen on a 1-3 year timeframe. For AI customers, there's already specialized cards like H100.

mcny 3y ago
> For AI customers, there's already specialized cards like H100 I don't know anything about machine learning but I think that is the question. Isn't nVidia professional tier like the H100 rare and expensive? I remember someone here on HN started a sharing club where they said they'd pool resources together to buy these things and allow people to basically timeshare or rent them? Is this like the situation where nVidia is basically writing your code for you by providing abstractions like it does with nVidia hair works and things like that on the gaming side? Would Intel have to basically develop its own answer to CUDA or could it gang up with AMD on this?
- nvm0n2 3y ago
  Yes H100s are very expensive (think ~$40k per unit), and very hard to obtain. For this reason they're basically all sent to cloud operators who can sign large bulk deals up front and then rent them out. At that point rubber really hits the road because it's where the hardware is basically auctioned off on an hour by hour basis, so you can easily pay $40k per month for access to the cards because there's so much money flooding into AI right now. This is why OpenAI seem to have burned through 10s of billions of dollars of Azure compute credits in, like, a few years. Yes CUDA is basically a giant developer platform made by NVIDIA for their cards. It's libraries, tools, languages, tutorials, and other ecosystem stuff like the fact that lots of people use it. Intel and AMD might or might not need their own answer to it. They need at least parts of an answer but CUDA is a lot more than AI, so if that's all you care about, you can skip parts of it. Now AMD has been trying to develop a CUDA competitor for a long time but it's clear either their heart isn't in it, or they just really struggle to attract skilled dev plaform people. ROCm seems to suck so hard people would rather pay NVIDIA's monopoly pricing than deal with it, which is impressively bad. Of course the reason NVIDIA is in this position is that they supported all kinds of R&D with their CUDA effort, and it turned out to be AI that hit big. If you just implements the bits of CUDA that seem most useful you'll always be behind.
macrolime 3y ago
The H100 is orders of magnitude too expensive for 99% of people who wants to do AI inference.
Wytwwww 3y ago
> For AI customers, there's already specialized cards like H100. Which is extremely expensive because Nvidia can charger whatever they want because they effectively are a monopoly. Since power or per card performance doesn't really matter that much yet Intel could in theory compete only on price as long as they manage to cobble something at least semi-decent together.

mcny 3y ago

> For AI customers, there's already specialized cards like H100 I don't know anything about machine learning but I think that is the question. Isn't nVidia professional tier like the H100 rare and expensive? I remember someone here on HN started a sharing club where they said they'd pool resources together to buy these things and allow people to basically timeshare or rent them? Is this like the situation where nVidia is basically writing your code for you by providing abstractions like it does with nVidia hair works and things like that on the gaming side? Would Intel have to basically develop its own answer to CUDA or could it gang up with AMD on this?

nvm0n2 3y ago
Yes H100s are very expensive (think ~$40k per unit), and very hard to obtain. For this reason they're basically all sent to cloud operators who can sign large bulk deals up front and then rent them out. At that point rubber really hits the road because it's where the hardware is basically auctioned off on an hour by hour basis, so you can easily pay $40k per month for access to the cards because there's so much money flooding into AI right now. This is why OpenAI seem to have burned through 10s of billions of dollars of Azure compute credits in, like, a few years. Yes CUDA is basically a giant developer platform made by NVIDIA for their cards. It's libraries, tools, languages, tutorials, and other ecosystem stuff like the fact that lots of people use it. Intel and AMD might or might not need their own answer to it. They need at least parts of an answer but CUDA is a lot more than AI, so if that's all you care about, you can skip parts of it. Now AMD has been trying to develop a CUDA competitor for a long time but it's clear either their heart isn't in it, or they just really struggle to attract skilled dev plaform people. ROCm seems to suck so hard people would rather pay NVIDIA's monopoly pricing than deal with it, which is impressively bad. Of course the reason NVIDIA is in this position is that they supported all kinds of R&D with their CUDA effort, and it turned out to be AI that hit big. If you just implements the bits of CUDA that seem most useful you'll always be behind.

macrolime 3y ago

The H100 is orders of magnitude too expensive for 99% of people who wants to do AI inference.

Wytwwww 3y ago

> For AI customers, there's already specialized cards like H100. Which is extremely expensive because Nvidia can charger whatever they want because they effectively are a monopoly. Since power or per card performance doesn't really matter that much yet Intel could in theory compete only on price as long as they manage to cobble something at least semi-decent together.