4 ms·

The market (and selling price) is reflecting the perceived value of nvidia's solution vs AMDs - comprehensively including tooling, software, TCO and managabilit

The market (and selling price) is reflecting the perceived value of nvidia's solution vs AMDs - comprehensively including tooling, software, TCO and managability. Also curious how many companies are dropping that much money on those kind of accelerators just to run 8x 7B param models in parallel... You're also talking about being able to train a 14B model on a single accelerator. I'd be curious to see how "full-accelerator train and inferrence" workloads would look ie: Training a 14B param model then inferrence throughput on a 4x14B workload. AMD (and almost every other inferrence claim maker so far... intel and apple specifically) have consistently cherry picked the benchmarks to claim a win over, and ignored the remainder which all show nvidia in the lead - and they've used mid-gen comparison models as many commenters here pointed out in this article.

fvv 2y ago
mi300x win in some inference workloads, h100 win in training and some others inference workloads ( fp8 inference with tensorRT-llm , rocm is young but is growing fast ) in a single system ( 8x accelerators ) LLMs, mi300x has very competitive inference TCO vs h100 . also : AMD Instinct MI300X Offers The Best Price To Performance on GPT-4 According To Microsoft, Red Team On-Track For 100x Perf/Watt By 2027 https://wccftech.com/amd-instinct-mi300x-best-price-performance-gpt-4-microsoft-on-track-100x-perf-watt-2027/ https://wccftech.com/amd-instinct-mi300x-best-price-performa...
- lostmsu 2y ago
  wccftech is an untrustworthy source.
  - fvv 2y ago
    the article contain a quote from satya nadella : https://x.com/ryanshrout/status/1792953227841015897 https://x.com/ryanshrout/status/1792953227841015897
    - lostmsu 2y ago
      None of the mentioned claims from the article are confirmed there.
fvv 2y ago
the market and the selling price also includes sales strategies, penetrating a sector dominated by a strong player with somewhat "smart" sales strategies *1 and with a growing but certainly less mature product ( expecially software ), it requires suitable pricing and allocation strategies 1. https://www.techspot.com/news/102056-nvidia-allegedly-punishing-customers-shipment-delays-leads-former.html https://www.techspot.com/news/102056-nvidia-allegedly-punish...
- fvv 2y ago
  the price of h100 reflects and reflected the fact that there is a total monopoly in the training sector, amd is successfully attacking the inference sector, increasing its advantage with mi325 and aiming for training from 2025 with mi350 (and Infinity Fabric interconnect and other types of interconnection that are arriving for the various topologies), which will probably have an advantage over blackwell, and then fall back against rubin and come back ahead against mi400, at least, this is what it seems, and as long as the rocm continues to improve. Personally I am happy to see some competition in the sector and especially on open source software
- paulmd 2y ago
  This stuff is the actual reason nvidia is under antitrust investigation. boo boo, a GTX 670 that cost you $399 in 2012 now costs $599 - grow up, do the inflation calculation, and realize you’re being a child. gamers get the best deal on bulk silicon on the planet, R&D subsidized by enterprise, fantastic blue-sky research that takes years for competitors to (not even) match, and it’s still never enough. ”Gamers” have justified every single cliche and stereotype over the last 5 years, absolutely inveterate manbabies. (Hardware Unboxed put out a video today with the headline+caption combo “are gamers entitled”/“are GeForce gpus gross”, and that’s what passes for reasoned discourse among the most popular channels. They’ve been trading segments back and forth with GN that are just absolute “how bad is nvidia” “real bad, but what do you guys think???” tier shit, lmao. https://i.imgur.com/98x0F1H.png https://i.imgur.com/98x0F1H.png this stuff is real shit, nvidia has been leaning on partners to maintain their segmentation, micromanaging shipment release to maintain price levels (cartel behavior), punishing customers and suppliers with “you know what will happen if you cross us”, literally putting it in writing with GPP (big mistake), playing fuck fuck games with not letting the drivers be run in a datacenter, etc. You see how that’s a little different than a gpu going from an inflation-adjusted $570 to $599 over 10 years? (And what’s worse the competition can’t even keep that much, they’re falling off even harder now that Moores law has really kicked the bucket and they have to do architectural work every gen just to make progress, instead of getting free shrinks etc… let alone having to develop software! /gasp) In entirely unrelated news… gigabyte suddenly has a 4070 ti super with a blower cooler. Oh, and it’s single-slot with end-fire power connector. All three forbidden features at once - very subtle, extremely law-abiding. https://videocardz.com/newz/gigabyte-unveils-geforce-rtx-4070-ti-super-ai-top-graphics-card-with-blower-style-cooler https://videocardz.com/newz/gigabyte-unveils-geforce-rtx-407... and literally gamers can’t help but think this whole ftc case is all about themselves anyway…
fvv 2y ago
mi300x production is ramping , in latest earning report lisa su said 1H2024 is production capped , 2h2024 have increased production ( and still have some to sell ), thanks probably to cowos and hbm3/(e?) supply improved large orders for those accelerators are placed months ahead meanwhile mi300x on microsoft are fully booked... https://techcommunity.microsoft.com/t5/azure-high-performance-computing/introducing-the-new-azure-ai-infrastructure-vm-series-nd-mi300x/ba-p/4145152 https://techcommunity.microsoft.com/t5/azure-high-performanc... "Scalable AI infrastructure running the capable OpenAI models These VMs, and the software that powers them, were purpose-built for our own Azure AI services production workloads. We have already optimized the most capable natural language model in the world, GPT-4 Turbo, for these VMs. ND MI300X v5 VMs offer leading cost performance for popular OpenAI and open-source models."

fvv 2y ago

mi300x win in some inference workloads, h100 win in training and some others inference workloads ( fp8 inference with tensorRT-llm , rocm is young but is growing fast ) in a single system ( 8x accelerators ) LLMs, mi300x has very competitive inference TCO vs h100 . also : AMD Instinct MI300X Offers The Best Price To Performance on GPT-4 According To Microsoft, Red Team On-Track For 100x Perf/Watt By 2027 https://wccftech.com/amd-instinct-mi300x-best-price-performance-gpt-4-microsoft-on-track-100x-perf-watt-2027/ https://wccftech.com/amd-instinct-mi300x-best-price-performa...

lostmsu 2y ago
wccftech is an untrustworthy source.
- fvv 2y ago
  the article contain a quote from satya nadella : https://x.com/ryanshrout/status/1792953227841015897 https://x.com/ryanshrout/status/1792953227841015897
  - lostmsu 2y ago
    None of the mentioned claims from the article are confirmed there.

fvv 2y ago

the market and the selling price also includes sales strategies, penetrating a sector dominated by a strong player with somewhat "smart" sales strategies *1 and with a growing but certainly less mature product ( expecially software ), it requires suitable pricing and allocation strategies 1. https://www.techspot.com/news/102056-nvidia-allegedly-punishing-customers-shipment-delays-leads-former.html https://www.techspot.com/news/102056-nvidia-allegedly-punish...

fvv 2y ago
the price of h100 reflects and reflected the fact that there is a total monopoly in the training sector, amd is successfully attacking the inference sector, increasing its advantage with mi325 and aiming for training from 2025 with mi350 (and Infinity Fabric interconnect and other types of interconnection that are arriving for the various topologies), which will probably have an advantage over blackwell, and then fall back against rubin and come back ahead against mi400, at least, this is what it seems, and as long as the rocm continues to improve. Personally I am happy to see some competition in the sector and especially on open source software
paulmd 2y ago
This stuff is the actual reason nvidia is under antitrust investigation. boo boo, a GTX 670 that cost you $399 in 2012 now costs $599 - grow up, do the inflation calculation, and realize you’re being a child. gamers get the best deal on bulk silicon on the planet, R&D subsidized by enterprise, fantastic blue-sky research that takes years for competitors to (not even) match, and it’s still never enough. ”Gamers” have justified every single cliche and stereotype over the last 5 years, absolutely inveterate manbabies. (Hardware Unboxed put out a video today with the headline+caption combo “are gamers entitled”/“are GeForce gpus gross”, and that’s what passes for reasoned discourse among the most popular channels. They’ve been trading segments back and forth with GN that are just absolute “how bad is nvidia” “real bad, but what do you guys think???” tier shit, lmao. https://i.imgur.com/98x0F1H.png https://i.imgur.com/98x0F1H.png this stuff is real shit, nvidia has been leaning on partners to maintain their segmentation, micromanaging shipment release to maintain price levels (cartel behavior), punishing customers and suppliers with “you know what will happen if you cross us”, literally putting it in writing with GPP (big mistake), playing fuck fuck games with not letting the drivers be run in a datacenter, etc. You see how that’s a little different than a gpu going from an inflation-adjusted $570 to $599 over 10 years? (And what’s worse the competition can’t even keep that much, they’re falling off even harder now that Moores law has really kicked the bucket and they have to do architectural work every gen just to make progress, instead of getting free shrinks etc… let alone having to develop software! /gasp) In entirely unrelated news… gigabyte suddenly has a 4070 ti super with a blower cooler. Oh, and it’s single-slot with end-fire power connector. All three forbidden features at once - very subtle, extremely law-abiding. https://videocardz.com/newz/gigabyte-unveils-geforce-rtx-4070-ti-super-ai-top-graphics-card-with-blower-style-cooler https://videocardz.com/newz/gigabyte-unveils-geforce-rtx-407... and literally gamers can’t help but think this whole ftc case is all about themselves anyway…

fvv 2y ago

mi300x production is ramping , in latest earning report lisa su said 1H2024 is production capped , 2h2024 have increased production ( and still have some to sell ), thanks probably to cowos and hbm3/(e?) supply improved large orders for those accelerators are placed months ahead meanwhile mi300x on microsoft are fully booked... https://techcommunity.microsoft.com/t5/azure-high-performance-computing/introducing-the-new-azure-ai-infrastructure-vm-series-nd-mi300x/ba-p/4145152 https://techcommunity.microsoft.com/t5/azure-high-performanc... "Scalable AI infrastructure running the capable OpenAI models These VMs, and the software that powers them, were purpose-built for our own Azure AI services production workloads. We have already optimized the most capable natural language model in the world, GPT-4 Turbo, for these VMs. ND MI300X v5 VMs offer leading cost performance for popular OpenAI and open-source models."