3 ms·

I think it leaves them with an option nVidia wants right? Paying them more for the VRAM.

by izacus 3y ago

cyanydeez 3y ago
right. it's bizzare watching tech heads confused about price segregation like it's some phantom force and not one of the glorious tools of monopolies and price gouging.
michaelt 3y ago
Even that, nvidia doesn't make easy. The RTX 4090 is so huge and power-hungry you can barely fit two into a case, let alone any more than that. And even if you replace your case, and your power supply (and maybe your motherboard and CPU too, gotta have enough PCIe lanes) you'll still only have 48GB of vram. OK, so we can't use an RTX GPU. Good luck even understanding the rest of nvidia's product range. The A40? "The World’s Most Powerful Data Center GPU for Visual Computing". V100? "The most advanced data center GPU ever built". H100? "Unprecedented performance, scalability, and security for every data center". A100? "Unprecedented acceleration at every scale to power the world’s highest-performing elastic data centers". H200? "The world’s most powerful GPU" L4? "The breakthrough universal accelerator". Good luck figuring out who sells them in your country, or if they're in stock. Oh, and remember to read the entire spec in excruciating detail. That $2700 L4 24GB is slower than an RTX 4090. Think you'll launch something in the cloud? With Google Cloud half the cards are only available in some regions - and even if you're in a region that offers a given GPU, maybe there's a shortage and support tells you to just keep requesting GPUs repeatedly until you get lucky.
- MichaelZuo 3y ago
  Nvidia specifically indicates the VRAM size and whether the product is suitable, so you pick the ones with enough to satisfy your expected needs. What’s there to not understand?
  - beeboobaa 3y ago
    That VRAM isn't everything, apparently.
  - michaelt 3y ago
    * Is a given product their most advanced, or was it merely their most advanced when the marketing copy was written 5 years ago? * Is the product actually available for purchase today, and at what price? * In what form factors - Full height? Half height? Two slots? Three? SXM? Does it have a fan built in? * What is the relative performance? Half the time the specs all quote different numbers. Why does one product quote 'single precision performance' and 'rt core performance' and 'tensor performance' while another quotes the 'tensor cores' and 'shader cores' and another quotes the 'FP64', 'FP32', 'FP16', 'BFLOAT16', 'TF32' and 'INT8'? And don't imagine you're going to get away with ignoring those specs. A $2700 L4 24GB is slower than an RTX 4090, for example, because it's for power-efficient servers or something.
    - MichaelZuo 3y ago
      Most of this information is indicated on their website or online spec sheets? Like I said Nvidia themselves indicate whether it’s suitable. And if your still confused or doubt the accuracy, or don’t want to spend any time reading spec sheets, I imagine the sales channel folks will be able to guarantee it in writing for a fee.
      - michaelt 3y ago
        > Most of this information is indicated on their website or online spec sheets? You'd think so, wouldn't you? But nvidia is not that smart. They have decided that the GPUs should be split over at least three different pages, and those pages should be camouflaged. I can only assume the marketing team are judged based on time spent on site or number of pages viewed, rather than on sales made. When you go to the home page, point to products and click on "NVIDIA RTX / Quadro" you might expect to find the RTX 4090 and some Quadro products. You will in fact find neither - the RTX 4090 is under 'GeForce' and the 'Quadro' brand is no longer used. Maybe on the home page you choose "NVIDIA RTX-Powered AI Workstations" - that'll give you a list of their workstation-suitable cards for ML, right? No, that page contains no products at all. The only option is to Find A Partner which links you to a bunch of partners - several of whom do not in fact sell AI Workstations. Other partners sell AI workstations... without GPUs. As far as HP is concerned, $4000 only gets you the base model workstation, with 32GB of RAM and a 1TB hard disk. If you want GPUs with that, you'll need to call the sales team, who might perhaps deign to sell you one. Or perhaps you're at nvidia's home page, and you're looking for data centre grade GPUs? For that product page, simply choose whether you mean the DGX, EGX, IGX, HGX, MGX or OVX platform? But they make it very easy to find the keynote speech by His Excellency Omar Sultan Al Olama on the latest breakthroughs in AI. He does, in fairness, have a very appropriate surname.
- gymbeaux 3y ago
  Yeah I went down the rabbit hole of trying to buy a GPU for local model training and execution about a year ago. My takeaway was “anything that starts with A is too expensive and often trades blows with a 3090 or 4090 anyway”. I ended up with a used 3090. 24GB of VRAM, which is more than some of the A series GPUs, and it was around $700 on eBay at the time. Now apparently they’re around a grand. I had to upgrade to a 1000W PSU for the 3090 because the 3090s will occasionally voltage spike and trigger OCP on power supplies. TDP is supposed to be like 350W but for a slip second it might pull around 600-700W. Plenty of complaints about it on Reddit and elsewhere. It’s a beast but it cooks my home office when I use it. Cest la vie.

I think it leaves them with an option nVidia wants right? Paying them more for the VRAM.

by izacus 3y ago

I think it leaves them with an option nVidia wants right? Paying them more for the VRAM.

right. it's bizzare watching tech heads confused about price segregation like it's some phantom force and not one of the glorious tools of monopolies and price gouging.

michaelt 3y ago

Even that, nvidia doesn't make easy. The RTX 4090 is so huge and power-hungry you can barely fit two into a case, let alone any more than that. And even if you replace your case, and your power supply (and maybe your motherboard and CPU too, gotta have enough PCIe lanes) you'll still only have 48GB of vram. OK, so we can't use an RTX GPU. Good luck even understanding the rest of nvidia's product range. The A40? "The World’s Most Powerful Data Center GPU for Visual Computing". V100? "The most advanced data center GPU ever built". H100? "Unprecedented performance, scalability, and security for every data center". A100? "Unprecedented acceleration at every scale to power the world’s highest-performing elastic data centers". H200? "The world’s most powerful GPU" L4? "The breakthrough universal accelerator". Good luck figuring out who sells them in your country, or if they're in stock. Oh, and remember to read the entire spec in excruciating detail. That $2700 L4 24GB is slower than an RTX 4090. Think you'll launch something in the cloud? With Google Cloud half the cards are only available in some regions - and even if you're in a region that offers a given GPU, maybe there's a shortage and support tells you to just keep requesting GPUs repeatedly until you get lucky.

MichaelZuo 3y ago
Nvidia specifically indicates the VRAM size and whether the product is suitable, so you pick the ones with enough to satisfy your expected needs. What’s there to not understand?
- beeboobaa 3y ago
  That VRAM isn't everything, apparently.
- michaelt 3y ago
  * Is a given product their most advanced, or was it merely their most advanced when the marketing copy was written 5 years ago? * Is the product actually available for purchase today, and at what price? * In what form factors - Full height? Half height? Two slots? Three? SXM? Does it have a fan built in? * What is the relative performance? Half the time the specs all quote different numbers. Why does one product quote 'single precision performance' and 'rt core performance' and 'tensor performance' while another quotes the 'tensor cores' and 'shader cores' and another quotes the 'FP64', 'FP32', 'FP16', 'BFLOAT16', 'TF32' and 'INT8'? And don't imagine you're going to get away with ignoring those specs. A $2700 L4 24GB is slower than an RTX 4090, for example, because it's for power-efficient servers or something.
  - MichaelZuo 3y ago
    Most of this information is indicated on their website or online spec sheets? Like I said Nvidia themselves indicate whether it’s suitable. And if your still confused or doubt the accuracy, or don’t want to spend any time reading spec sheets, I imagine the sales channel folks will be able to guarantee it in writing for a fee.
    - michaelt 3y ago
      > Most of this information is indicated on their website or online spec sheets? You'd think so, wouldn't you? But nvidia is not that smart. They have decided that the GPUs should be split over at least three different pages, and those pages should be camouflaged. I can only assume the marketing team are judged based on time spent on site or number of pages viewed, rather than on sales made. When you go to the home page, point to products and click on "NVIDIA RTX / Quadro" you might expect to find the RTX 4090 and some Quadro products. You will in fact find neither - the RTX 4090 is under 'GeForce' and the 'Quadro' brand is no longer used. Maybe on the home page you choose "NVIDIA RTX-Powered AI Workstations" - that'll give you a list of their workstation-suitable cards for ML, right? No, that page contains no products at all. The only option is to Find A Partner which links you to a bunch of partners - several of whom do not in fact sell AI Workstations. Other partners sell AI workstations... without GPUs. As far as HP is concerned, $4000 only gets you the base model workstation, with 32GB of RAM and a 1TB hard disk. If you want GPUs with that, you'll need to call the sales team, who might perhaps deign to sell you one. Or perhaps you're at nvidia's home page, and you're looking for data centre grade GPUs? For that product page, simply choose whether you mean the DGX, EGX, IGX, HGX, MGX or OVX platform? But they make it very easy to find the keynote speech by His Excellency Omar Sultan Al Olama on the latest breakthroughs in AI. He does, in fairness, have a very appropriate surname.
gymbeaux 3y ago
Yeah I went down the rabbit hole of trying to buy a GPU for local model training and execution about a year ago. My takeaway was “anything that starts with A is too expensive and often trades blows with a 3090 or 4090 anyway”. I ended up with a used 3090. 24GB of VRAM, which is more than some of the A series GPUs, and it was around $700 on eBay at the time. Now apparently they’re around a grand. I had to upgrade to a 1000W PSU for the 3090 because the 3090s will occasionally voltage spike and trigger OCP on power supplies. TDP is supposed to be like 350W but for a slip second it might pull around 600-700W. Plenty of complaints about it on Reddit and elsewhere. It’s a beast but it cooks my home office when I use it. Cest la vie.