3 ms·

Wait for just 2 days, 17 hours and 18 minutes - and NVIDIA will give you (rumored) 96GB of RAM inside GeForce RTX 4090 TI: https://www.nvidia.com/en-eu/geforce/

by pulse7 3y ago

Wait for just 2 days, 17 hours and 18 minutes - and NVIDIA will give you (rumored) 96GB of RAM inside GeForce RTX 4090 TI: https://www.nvidia.com/en-eu/geforce/special-event/ https://www.nvidia.com/en-eu/geforce/special-event/

doctorpangloss 3y ago
There is no chance the 4090 TI is going to ship with 96GB of VRAM. And besides, it has no NVLink, so it's a stranded asset. It would appear the poster misread reports about the L2 cache (96MB) for the VRAM (24GB).
- cjbprime 3y ago
  The lack of nvlink doesn't matter much for LLMs, which can work on distinct layers of the model on completely independent GPUs.
com2kid 3y ago
Doesn't fit in a backpack. Can't take it to a coffee shop. Heck it involves building an entire custom PC around it. Sure the m3 with 128GB of RAM costs $7k, which is an absurd number, but that will run high quality LLMs that can have sci-fi level intelligent conversations. Not a bad price for a laptop capable of doing things that was literally "haha that'll never happen" levels of unimaginable 5 years ago.
TillE 3y ago
I can't find even the sketchiest rumor claiming 96GB of VRAM on a 4090 Ti. That would be more than an H100, which costs around $30k.
- nomel 3y ago
  Agreed. Estimates put the H100 BOM at around $3k. If they do offer that much memory, they're going to implement some way to make it unattractive, to hold onto that sweet 800% margin [1]. 1. https://www.tomshardware.com/news/nvidia-makes-1000-profit-on-h100-gpus-report https://www.tomshardware.com/news/nvidia-makes-1000-profit-o...
  - acdha 3y ago
    I’d want a better source than someone on Twitter summarizing an unspecified report by a financial analyst before repeating that margin claim. That sounds like the people who claim an iPhone has 400% markup because they added up the bulk prices for the major components and then assumed R&D, assembly, testing, distribution, software development, and support all cost $0.
    - wtallis 3y ago
      > and then assumed R&D, assembly, testing, distribution, software development, and support all cost $0. That's literally what BOM means: bill of materials. It's not supposed to include any of those other costs. BOM is not always the right kind of cost to be talking about, but in a case like this where you're talking about just swapping out the memory on an existing product, BOM accounts for all the costs that change significantly.
      - acdha 3y ago
        Yes, now can I direct your attention to the term “margin” used in the comment I was replying to? It is common for people who are looking for the sweet upvotes from a “this company is ripping everyone off” spin to present this as if “retail sales price - BOM = UNJUST PROFIT!!!!!!”. The source is a blog post based on a tweet by one person who does not work in the industry who is repeating another unverifiable claim by someone who does not work in the industry. The term “BOM” is not mentioned anywhere in that, and none of this has any details which could be reviewed for accuracy. The blog post really highlights how effective those headlines are at getting clicks but not informing readers: it leads with “nearly 1,000%” because that sounds even more rage bait-y than the already high “823%”, and that's clearly the impression people take away from the story even though starting with the second paragraph it makes it clear that this is a very fuzzy claim which is likely significant understating the true cost for Nvidia to make the hardware as sold or the other factors I also mentioned. Finally, “swapping out the memory on an existing product” can mean “we replaced part A with larger part B” or it can mean “we upgraded a controller, added additional channels, and redesigned the physical form factor to provide better mechanical and thermal capacity for the extra chips”. That's why those extra details are so important since it's otherwise easy for someone to make a tweet which sounds like they know what they're talking about but is in fact leaving out a lot of important context.
sliken 3y ago
Definitely a nice GPU, but considering airflow, power use, heat generated and the like it's more like a GPU with a PC attached, not so much a PC with a GPU attached. Not very portable either.

by pulse7 3y ago

doctorpangloss 3y ago

There is no chance the 4090 TI is going to ship with 96GB of VRAM. And besides, it has no NVLink, so it's a stranded asset. It would appear the poster misread reports about the L2 cache (96MB) for the VRAM (24GB).

cjbprime 3y ago
The lack of nvlink doesn't matter much for LLMs, which can work on distinct layers of the model on completely independent GPUs.

com2kid 3y ago

Doesn't fit in a backpack. Can't take it to a coffee shop. Heck it involves building an entire custom PC around it. Sure the m3 with 128GB of RAM costs $7k, which is an absurd number, but that will run high quality LLMs that can have sci-fi level intelligent conversations. Not a bad price for a laptop capable of doing things that was literally "haha that'll never happen" levels of unimaginable 5 years ago.

TillE 3y ago

I can't find even the sketchiest rumor claiming 96GB of VRAM on a 4090 Ti. That would be more than an H100, which costs around $30k.

nomel 3y ago
Agreed. Estimates put the H100 BOM at around $3k. If they do offer that much memory, they're going to implement some way to make it unattractive, to hold onto that sweet 800% margin [1]. 1. https://www.tomshardware.com/news/nvidia-makes-1000-profit-on-h100-gpus-report https://www.tomshardware.com/news/nvidia-makes-1000-profit-o...
- acdha 3y ago
  I’d want a better source than someone on Twitter summarizing an unspecified report by a financial analyst before repeating that margin claim. That sounds like the people who claim an iPhone has 400% markup because they added up the bulk prices for the major components and then assumed R&D, assembly, testing, distribution, software development, and support all cost $0.
  - wtallis 3y ago
    > and then assumed R&D, assembly, testing, distribution, software development, and support all cost $0. That's literally what BOM means: bill of materials. It's not supposed to include any of those other costs. BOM is not always the right kind of cost to be talking about, but in a case like this where you're talking about just swapping out the memory on an existing product, BOM accounts for all the costs that change significantly.
    - acdha 3y ago
      Yes, now can I direct your attention to the term “margin” used in the comment I was replying to? It is common for people who are looking for the sweet upvotes from a “this company is ripping everyone off” spin to present this as if “retail sales price - BOM = UNJUST PROFIT!!!!!!”. The source is a blog post based on a tweet by one person who does not work in the industry who is repeating another unverifiable claim by someone who does not work in the industry. The term “BOM” is not mentioned anywhere in that, and none of this has any details which could be reviewed for accuracy. The blog post really highlights how effective those headlines are at getting clicks but not informing readers: it leads with “nearly 1,000%” because that sounds even more rage bait-y than the already high “823%”, and that's clearly the impression people take away from the story even though starting with the second paragraph it makes it clear that this is a very fuzzy claim which is likely significant understating the true cost for Nvidia to make the hardware as sold or the other factors I also mentioned. Finally, “swapping out the memory on an existing product” can mean “we replaced part A with larger part B” or it can mean “we upgraded a controller, added additional channels, and redesigned the physical form factor to provide better mechanical and thermal capacity for the extra chips”. That's why those extra details are so important since it's otherwise easy for someone to make a tweet which sounds like they know what they're talking about but is in fact leaving out a lot of important context.

sliken 3y ago

Definitely a nice GPU, but considering airflow, power use, heat generated and the like it's more like a GPU with a PC attached, not so much a PC with a GPU attached. Not very portable either.