6 ms·

I'm genuinely rooting for AMD to develop a competitive alternative to NVIDIA. Currently, NVIDIA dominates the machine learning landscape, and there doesn't seem

by bratao 3y ago

I'm genuinely rooting for AMD to develop a competitive alternative to NVIDIA. Currently, NVIDIA dominates the machine learning landscape, and there doesn't seem to be a justifiable reason for the price discrepancy between the RTX 4090 and the A100.

ChuckNorris89 3y ago
>a justifiable reason for the price discrepancy between the RTX 4090 and the A100 1. they're a publicly listed company operating in a free market in an industry they themselves helped develop who's purpose is to make returns for their investors, not be liked by the public, they don't need to justify their pricing to their buyers. They're not selling essentials for survival like insulin, baby formula, or housing, they can charge as much as the market will bear for their consumer electronics products. Don't like the pricing? Don't buy it. Simple. Buy from the competition instead or older generations off the second hand market that fit your budget. 2. the price justification is that cutting edge silicon is and will always be in short supply, and buyers of the silicon in A100 form, like datacenters, use it to make money, therefore it's an investment that will yield returns, therefore they can justify spending way more money to outbid the gamers who buy the same silicon in RTX 4090 form and don't use it to make money but use it to play games therefore for them the product is worth less and it makes Nvidia smaller margins than what selling it to datacenters can. It's basic price segmentation that's been going on for decades. I also don't like the GPU pricing situation but that's the market reality I can't change and downvoting the messenger won't change it either. My 2 cents.
- ActorNightly 3y ago
  Yeah but the issue is that they can make more money if they actually did affordable ML hardware and didn't purposefully gimp the gaming cards. LLms running on standalone affordable boxes could become a ubiquitous home accessory, not to mention the potential for things like being able to leverage learning from widespread use to retrain the models, like Tesla does with their fleet.
  - ChuckNorris89 3y ago
    >Yeah but the issue is that they can make more money if they actually did affordable ML hardware and didn't purposefully gimp the gaming cards. It's a dick move but it's not illegal (at least not now). Their purpose is to make greater returns for their investors, not make affordable ML hardware for every consumer. >LLms running on standalone affordable boxes could become a ubiquitous home accessory They could be, but why is it Nvidia's fault if they don't happen? They're in the business of making money, not fulfilling dreams, and currently they can barely fulfill their orders for the datacenter customers. Consumers could move the LLM stuff on cheaper Apple HW or Intel or AMD SoCs if they can't outbid the datacenter companies for Nvidia silicone. Seems like a market opening for Nvidia's competitors to price cut them. If they don't exploit it and let Nvidia dominate, it's their fault. It's not Nvidia's fault their competitors were asleep at the wheel since they launched CUDA in 2007 and were helping researchers put their GPUs to use for parallel computing for over 15 years.
    - gremlinsinc 3y ago
      If AMD were smart(er), their cpu's are awesome at least... They would team up with intel, google, apple, etc to create a machine learning open cuda alternative, such that all GPU's could essentially be used by ML software equally, I think Nvidia's real hold on AI is that most AI libs use CUDA, and nobody else can really compete when all the tools use X, it's like if firefox launched with their own different HTML syntax because HTML was proprietary, and every other browser did the same thing. Something like chrome : chromium dynamic would work wonders for bringing a sort of standard, not to mention more minds could create better and better implementations.
      - ornitorrincos 3y ago
        They did try, apple made OpenCL and released it to khronos to make a standard, but never got wide support(well, they got plenty of manufacturers on board, not so much users). By that time nvidia already had an advantage with CUDA and their opencl support always lagged.
      - deleted 3y ago
- scld 3y ago
  It's because you're responding as if OP said it wasn't justifiable legally. They're just saying that AMD is, in their estimation, making a bad business decision.
  - ChuckNorris89 3y ago
    That's was really not his point. His point was that Nvidia is price gouging and I explained him why they can get away with it since the current market conditions favor their products a lot. AMD has been consistently making bad business decisions in the GPU space for years now. What's new? They're pricing their hardware near NVidia but with less performance and way less features and poor track record, especially for the ML users.
    - wwtrv 3y ago
      Nvidia is price gouging because there is no credible competition. That's the entire explanation and there is nothing more to it.
      - ChuckNorris89 3y ago
        Why are people here continuously surprised by this and keep asking the same redundant questions: "Why does Nvidia charge so much?" I just don't get it. Thinking that Nvidia is some evil villain doing it to consumer out of spite, when they're just doing what any other company in their dominant position would do: charge as much as the market will bear. Apple also doesn't have to charge you $200 for configuring the 512GB SSD over the 256 SSD which only costs them an extra $5 NAND chip, but they do it because they can. So does Nvidia and any other company in a dominant position with virtually no competition.
    - The_Colonel 3y ago
      > His point was that Nvidia is price gouging You might be reading into their comment too much, they don't really blame NVidia, just wish for more competition. IMHO you're being downvoted because you come across as needlessly confrontational to an innocent comment.
deleted 3y ago
[deleted]
brucethemoose2 3y ago
Its not so black and white. The A100 is exponentially more difficult to assemble even though it is older. The silicon is far more specialized. You pay a price for such a "fat" node and server hardware guarantees. At the same time, the cost is outrageous. Its not a low volume product. Also, a 48GB 4090 (or 3090) would be trivial. So would a 48GB 7900. Its not done for purely anticompetitive reasons, one that AMD and Nvidia are unfortunately happy to go along with.
- BaculumMeumEst 3y ago
  > Also, a 48GB 4090 (or 3090) would be trivial. So would a 48GB 7900. Its not done for purely anticompetitive reasons, one that AMD and Nvidia are unfortunately happy to go along with. Is this why Intel started taking dGPU production more seriously in recent years?
  - deleted 3y ago
    [deleted]
- wmf 3y ago
  There kind of are 48 GB 4090s and 7900s; they're just under different names.
  - brucethemoose2 3y ago
    Yeah, but Quadros (and the AMD equivalents) are outrageously priced too. Nvidia and AMD used to "let" their manufacturers double up the vram on gaming cards, but no more. Its pure, collusive, anticompetitive market segmentation.
  - splitrocket 3y ago
    what names?
- densh 3y ago
  48Gb 4090 exists already, just with a different name, branding, and pricing strategy: https://www.nvidia.com/en-us/design-visualization/rtx-6000/ https://www.nvidia.com/en-us/design-visualization/rtx-6000/ 48Gb 3090: https://www.nvidia.com/en-us/design-visualization/rtx-a6000/ https://www.nvidia.com/en-us/design-visualization/rtx-a6000/
valyagolev 3y ago
comma.ai dude is making a whole company to achieve just that https://geohot.github.io/blog/jekyll/update/2023/05/24/the-tiny-corp-raised-5M.html https://geohot.github.io/blog/jekyll/update/2023/05/24/the-t...
- synthos 3y ago
  Interesting, seems like it's very similar to Modular's runtime portion of their business case
- paulmd 3y ago
  Nope, he got a look at the absolute state of ROCm and realized he was wasting his time. https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuecomment-1574383483 https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuec... One might speculate he’s perhaps pivoting to Intel. They’re not well-developed in application terms but that’s a piece he can develop, and actually with OneAPI that’s a lot of potential bang for the buck. And intel has actual ML accelerators and has some relatively powerful GPGPU stuff and is currently in a position of being forced to offer a lot of bang for buck to drive adoption, all of which makes sense for what he’s trying to do. But AMD wants you to basically write and debug their runtime for them and nah not worth it, after fighting the installer on the official system config and then filing a couple bugs for the demo apps reproducibly crashing the kernel it’s just not worth the time. ROCm is unserious even when you’re operating on supported hardware. This is the experience most people have with it.
  - neurostimulant 3y ago
    Follow up post: https://geohot.github.io/blog/jekyll/update/2023/06/07/a-dive-into-amds-drivers.html https://geohot.github.io/blog/jekyll/update/2023/06/07/a-div...
    - paulmd 3y ago
      Ah yes, simply be famous enough to get the closed internal build that actually works. You know, it doesn't surprise me that's what they're doing, because the HPC people aren't paying megabucks and then tolerating something that crashes the kernel... but people's problems/experiences aren't invalid either, and the delta is they're not running the same software. I don't like it but it makes sense.

by bratao 3y ago

ChuckNorris89 3y ago

>a justifiable reason for the price discrepancy between the RTX 4090 and the A100 1. they're a publicly listed company operating in a free market in an industry they themselves helped develop who's purpose is to make returns for their investors, not be liked by the public, they don't need to justify their pricing to their buyers. They're not selling essentials for survival like insulin, baby formula, or housing, they can charge as much as the market will bear for their consumer electronics products. Don't like the pricing? Don't buy it. Simple. Buy from the competition instead or older generations off the second hand market that fit your budget. 2. the price justification is that cutting edge silicon is and will always be in short supply, and buyers of the silicon in A100 form, like datacenters, use it to make money, therefore it's an investment that will yield returns, therefore they can justify spending way more money to outbid the gamers who buy the same silicon in RTX 4090 form and don't use it to make money but use it to play games therefore for them the product is worth less and it makes Nvidia smaller margins than what selling it to datacenters can. It's basic price segmentation that's been going on for decades. I also don't like the GPU pricing situation but that's the market reality I can't change and downvoting the messenger won't change it either. My 2 cents.

ActorNightly 3y ago
Yeah but the issue is that they can make more money if they actually did affordable ML hardware and didn't purposefully gimp the gaming cards. LLms running on standalone affordable boxes could become a ubiquitous home accessory, not to mention the potential for things like being able to leverage learning from widespread use to retrain the models, like Tesla does with their fleet.
- ChuckNorris89 3y ago
  >Yeah but the issue is that they can make more money if they actually did affordable ML hardware and didn't purposefully gimp the gaming cards. It's a dick move but it's not illegal (at least not now). Their purpose is to make greater returns for their investors, not make affordable ML hardware for every consumer. >LLms running on standalone affordable boxes could become a ubiquitous home accessory They could be, but why is it Nvidia's fault if they don't happen? They're in the business of making money, not fulfilling dreams, and currently they can barely fulfill their orders for the datacenter customers. Consumers could move the LLM stuff on cheaper Apple HW or Intel or AMD SoCs if they can't outbid the datacenter companies for Nvidia silicone. Seems like a market opening for Nvidia's competitors to price cut them. If they don't exploit it and let Nvidia dominate, it's their fault. It's not Nvidia's fault their competitors were asleep at the wheel since they launched CUDA in 2007 and were helping researchers put their GPUs to use for parallel computing for over 15 years.
  - gremlinsinc 3y ago
    If AMD were smart(er), their cpu's are awesome at least... They would team up with intel, google, apple, etc to create a machine learning open cuda alternative, such that all GPU's could essentially be used by ML software equally, I think Nvidia's real hold on AI is that most AI libs use CUDA, and nobody else can really compete when all the tools use X, it's like if firefox launched with their own different HTML syntax because HTML was proprietary, and every other browser did the same thing. Something like chrome : chromium dynamic would work wonders for bringing a sort of standard, not to mention more minds could create better and better implementations.
    - ornitorrincos 3y ago
      They did try, apple made OpenCL and released it to khronos to make a standard, but never got wide support(well, they got plenty of manufacturers on board, not so much users). By that time nvidia already had an advantage with CUDA and their opencl support always lagged.
    - deleted 3y ago
scld 3y ago
It's because you're responding as if OP said it wasn't justifiable legally. They're just saying that AMD is, in their estimation, making a bad business decision.
- ChuckNorris89 3y ago
  That's was really not his point. His point was that Nvidia is price gouging and I explained him why they can get away with it since the current market conditions favor their products a lot. AMD has been consistently making bad business decisions in the GPU space for years now. What's new? They're pricing their hardware near NVidia but with less performance and way less features and poor track record, especially for the ML users.
  - wwtrv 3y ago
    Nvidia is price gouging because there is no credible competition. That's the entire explanation and there is nothing more to it.
    - ChuckNorris89 3y ago
      Why are people here continuously surprised by this and keep asking the same redundant questions: "Why does Nvidia charge so much?" I just don't get it. Thinking that Nvidia is some evil villain doing it to consumer out of spite, when they're just doing what any other company in their dominant position would do: charge as much as the market will bear. Apple also doesn't have to charge you $200 for configuring the 512GB SSD over the 256 SSD which only costs them an extra $5 NAND chip, but they do it because they can. So does Nvidia and any other company in a dominant position with virtually no competition.
  - The_Colonel 3y ago
    > His point was that Nvidia is price gouging You might be reading into their comment too much, they don't really blame NVidia, just wish for more competition. IMHO you're being downvoted because you come across as needlessly confrontational to an innocent comment.

deleted 3y ago

[deleted]

brucethemoose2 3y ago

Its not so black and white. The A100 is exponentially more difficult to assemble even though it is older. The silicon is far more specialized. You pay a price for such a "fat" node and server hardware guarantees. At the same time, the cost is outrageous. Its not a low volume product. Also, a 48GB 4090 (or 3090) would be trivial. So would a 48GB 7900. Its not done for purely anticompetitive reasons, one that AMD and Nvidia are unfortunately happy to go along with.

BaculumMeumEst 3y ago
> Also, a 48GB 4090 (or 3090) would be trivial. So would a 48GB 7900. Its not done for purely anticompetitive reasons, one that AMD and Nvidia are unfortunately happy to go along with. Is this why Intel started taking dGPU production more seriously in recent years?
- deleted 3y ago
  [deleted]
wmf 3y ago
There kind of are 48 GB 4090s and 7900s; they're just under different names.
- brucethemoose2 3y ago
  Yeah, but Quadros (and the AMD equivalents) are outrageously priced too. Nvidia and AMD used to "let" their manufacturers double up the vram on gaming cards, but no more. Its pure, collusive, anticompetitive market segmentation.
- splitrocket 3y ago
  what names?
densh 3y ago
48Gb 4090 exists already, just with a different name, branding, and pricing strategy: https://www.nvidia.com/en-us/design-visualization/rtx-6000/ https://www.nvidia.com/en-us/design-visualization/rtx-6000/ 48Gb 3090: https://www.nvidia.com/en-us/design-visualization/rtx-a6000/ https://www.nvidia.com/en-us/design-visualization/rtx-a6000/

valyagolev 3y ago

comma.ai dude is making a whole company to achieve just that https://geohot.github.io/blog/jekyll/update/2023/05/24/the-tiny-corp-raised-5M.html https://geohot.github.io/blog/jekyll/update/2023/05/24/the-t...

synthos 3y ago
Interesting, seems like it's very similar to Modular's runtime portion of their business case
paulmd 3y ago
Nope, he got a look at the absolute state of ROCm and realized he was wasting his time. https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuecomment-1574383483 https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuec... One might speculate he’s perhaps pivoting to Intel. They’re not well-developed in application terms but that’s a piece he can develop, and actually with OneAPI that’s a lot of potential bang for the buck. And intel has actual ML accelerators and has some relatively powerful GPGPU stuff and is currently in a position of being forced to offer a lot of bang for buck to drive adoption, all of which makes sense for what he’s trying to do. But AMD wants you to basically write and debug their runtime for them and nah not worth it, after fighting the installer on the official system config and then filing a couple bugs for the demo apps reproducibly crashing the kernel it’s just not worth the time. ROCm is unserious even when you’re operating on supported hardware. This is the experience most people have with it.
- neurostimulant 3y ago
  Follow up post: https://geohot.github.io/blog/jekyll/update/2023/06/07/a-dive-into-amds-drivers.html https://geohot.github.io/blog/jekyll/update/2023/06/07/a-div...
  - paulmd 3y ago
    Ah yes, simply be famous enough to get the closed internal build that actually works. You know, it doesn't surprise me that's what they're doing, because the HPC people aren't paying megabucks and then tolerating something that crashes the kernel... but people's problems/experiences aren't invalid either, and the delta is they're not running the same software. I don't like it but it makes sense.