6 ms·

Aren’t their margins a product of producing better products? Market power implies it’s because nvidia is using methods other than better products to keep its p

by olliej 3y ago

Aren’t their margins a product of producing better products? Market power implies it’s because nvidia is using methods other than better products to keep its prices high.

LoganDark 3y ago
That's because they are. Ever since the mining crash Nvidia has been selling all its GPUs for exorbitant prices, much to the detriment of people who need the VRAM for machine learning. Oh well, guess you're buying the (checks notes) $13,000 A100, just because you need 3.3 times the VRAM of the $1600 4090 (which could be $5280 instead, if price corresponds exactly to VRAM).
- thrtythreeforty 3y ago
  At the risk of sounding like a broken record: Once others get their software shit together, they can have some of those profits. Nvidia has had a competent software game for a decade.
  - LoganDark 3y ago
    They still haven't gotten their hardware shit together. As a consumer I can't buy a PCIe card that slots into my system with more than 24GB of VRAM, whether it's from Nvidia, AMD, or Intel. Nvidia's prices will continue to be high while there is literally no competition. I guess before datacenter class cards can trickle down to the consumer, they do have to get their software shit together first.
    - kcb 3y ago
      https://www.bhphotovideo.com/c/product/1753962-REG/pny_vcnrtx6000ada_pb_rtx_6000_ada_generation.html/?ap=y&ap=y&smp=y&smp=y&smpm=ba_f2_lar&lsft=BI%3A6879&gad_source=1 https://www.bhphotovideo.com/c/product/1753962-REG/pny_vcnrt...
    - my123 3y ago
      NVIDIA and AMD both have workstation PCIe cards with more than 24GB or VRAM (W7900 from AMD and RTX 6000 Ada both have 48GB). They just want more money.
      - LoganDark 3y ago
        Cool. Where can I buy one? If I go to the page for the W7900, I can download the datasheet. I can look at the marketing pages. I can look at the specifications and download drivers. But I can't buy the card. If I look for one on eBay, they start at $4600, which is still far too expensive for 48GB. At that point, just buy an Apple Silicon machine and you can get far more memory than that for even cheaper. (M3 Max MacBook Pro with 96GB memory is just $4,000!) Alright, let's try the RTX A6000! Oop, it's $4,000. Cheaper than AMD, but wow! Oh, and I can only find these prices on eBay, you can't buy them direct from Nvidia as a consumer.
        NilMostChill 3y ago
        Just checking here but are you comparing a macbook with 96GB of machine RAM to a GPU with 48GB of on-card RAM? Because if so you can get a good pc with 96GB of machine ram for significantly less than $4000. Unless the claim is that an M3 Max MacBook has equivalent GPU type compute power as a dedicated GPU?
        o_x 3y ago
        It's probably because of the unified memory model - the GPU on the macbook can access the memory as if it was dedicated.
        NilMostChill 3y ago
        Interesting, i didn't know this existed.
        LoganDark 3y ago
        I'm comparing the cost of a GPU with 48GB VRAM to the cost of an Apple Silicon machine with at least 48GB of Unified Memory, yes. (The ASi machine having 96GB of memory instead is just because it's still cheaper than the card, so why not.) A normal PC with 96GB of regular RAM wouldn't allow you to use that RAM for GPU workloads, at least not with a dedicated GPU.
- amadeuspagel 3y ago
  Does the price of whatever SaaS you're working on correspond exactly to the RAM usage of a customer?
  - LoganDark 3y ago
    Well, my (totally made-up) calculation conservatively assumes that multiplying the VRAM multiplies the price by the same amount. In practice I don't think it's even that expensive to add more VRAM, since most of the rest of the card should stay the same?
est31 3y ago
Market power just means having a negotiating position that allows you to determine prices, regardless of how that negotiating position is obtained. It can either come from illicit means (say cartels for example), or from true innovation and investing into the right things. I think that both the customers of Nvidia as well as the competition are keen on replacing it (given the scale that many of the big customers work at, I'm sure they can afford the engineers to migrate the LLM models to ROCm if it were a viable alternative), but it seems that they cannot manage to make replacements fast enough. > The most interesting detail is that Nvidia managed to double sales without incurring additional costs
- mensetmanusman 3y ago
  market power also comes from a material log jam because we can't make GPUs fast enough to satisfy demand, so the wealthiest customers choose the price because the GPUs built essentially go to auction.
amadeuspagel 3y ago
"In economics, market power refers to the ability of a firm to influence the price at which it sells a product or service by manipulating either the supply or demand of the product or service to increase economic profit.[1] In other words, market power occurs if a firm does not face a perfectly elastic demand curve and can set its price (P) above marginal cost (MC) without losing revenue." https://en.wikipedia.org/wiki/Market_power https://en.wikipedia.org/wiki/Market_power

by olliej 3y ago

Aren’t their margins a product of producing better products? Market power implies it’s because nvidia is using methods other than better products to keep its prices high.

LoganDark 3y ago

That's because they are. Ever since the mining crash Nvidia has been selling all its GPUs for exorbitant prices, much to the detriment of people who need the VRAM for machine learning. Oh well, guess you're buying the (checks notes) $13,000 A100, just because you need 3.3 times the VRAM of the $1600 4090 (which could be $5280 instead, if price corresponds exactly to VRAM).

thrtythreeforty 3y ago
At the risk of sounding like a broken record: Once others get their software shit together, they can have some of those profits. Nvidia has had a competent software game for a decade.
- LoganDark 3y ago
  They still haven't gotten their hardware shit together. As a consumer I can't buy a PCIe card that slots into my system with more than 24GB of VRAM, whether it's from Nvidia, AMD, or Intel. Nvidia's prices will continue to be high while there is literally no competition. I guess before datacenter class cards can trickle down to the consumer, they do have to get their software shit together first.
  - kcb 3y ago
    https://www.bhphotovideo.com/c/product/1753962-REG/pny_vcnrtx6000ada_pb_rtx_6000_ada_generation.html/?ap=y&ap=y&smp=y&smp=y&smpm=ba_f2_lar&lsft=BI%3A6879&gad_source=1 https://www.bhphotovideo.com/c/product/1753962-REG/pny_vcnrt...
  - my123 3y ago
    NVIDIA and AMD both have workstation PCIe cards with more than 24GB or VRAM (W7900 from AMD and RTX 6000 Ada both have 48GB). They just want more money.
    - LoganDark 3y ago
      Cool. Where can I buy one? If I go to the page for the W7900, I can download the datasheet. I can look at the marketing pages. I can look at the specifications and download drivers. But I can't buy the card. If I look for one on eBay, they start at $4600, which is still far too expensive for 48GB. At that point, just buy an Apple Silicon machine and you can get far more memory than that for even cheaper. (M3 Max MacBook Pro with 96GB memory is just $4,000!) Alright, let's try the RTX A6000! Oop, it's $4,000. Cheaper than AMD, but wow! Oh, and I can only find these prices on eBay, you can't buy them direct from Nvidia as a consumer.
      - NilMostChill 3y ago
        Just checking here but are you comparing a macbook with 96GB of machine RAM to a GPU with 48GB of on-card RAM? Because if so you can get a good pc with 96GB of machine ram for significantly less than $4000. Unless the claim is that an M3 Max MacBook has equivalent GPU type compute power as a dedicated GPU?
        o_x 3y ago
        It's probably because of the unified memory model - the GPU on the macbook can access the memory as if it was dedicated.
        NilMostChill 3y ago
        Interesting, i didn't know this existed.
        LoganDark 3y ago
        I'm comparing the cost of a GPU with 48GB VRAM to the cost of an Apple Silicon machine with at least 48GB of Unified Memory, yes. (The ASi machine having 96GB of memory instead is just because it's still cheaper than the card, so why not.) A normal PC with 96GB of regular RAM wouldn't allow you to use that RAM for GPU workloads, at least not with a dedicated GPU.
amadeuspagel 3y ago
Does the price of whatever SaaS you're working on correspond exactly to the RAM usage of a customer?
- LoganDark 3y ago
  Well, my (totally made-up) calculation conservatively assumes that multiplying the VRAM multiplies the price by the same amount. In practice I don't think it's even that expensive to add more VRAM, since most of the rest of the card should stay the same?

est31 3y ago

Market power just means having a negotiating position that allows you to determine prices, regardless of how that negotiating position is obtained. It can either come from illicit means (say cartels for example), or from true innovation and investing into the right things. I think that both the customers of Nvidia as well as the competition are keen on replacing it (given the scale that many of the big customers work at, I'm sure they can afford the engineers to migrate the LLM models to ROCm if it were a viable alternative), but it seems that they cannot manage to make replacements fast enough. > The most interesting detail is that Nvidia managed to double sales without incurring additional costs

mensetmanusman 3y ago
market power also comes from a material log jam because we can't make GPUs fast enough to satisfy demand, so the wealthiest customers choose the price because the GPUs built essentially go to auction.

amadeuspagel 3y ago

"In economics, market power refers to the ability of a firm to influence the price at which it sells a product or service by manipulating either the supply or demand of the product or service to increase economic profit.[1] In other words, market power occurs if a firm does not face a perfectly elastic demand curve and can set its price (P) above marginal cost (MC) without losing revenue." https://en.wikipedia.org/wiki/Market_power https://en.wikipedia.org/wiki/Market_power