3 ms·

if chiplet is the silver bullet to die costs, I don't get why Nvidia won't go for it. After all, Nvidia has smart ppl too. There must be some design tradeoffs t

by russli1993 4y ago

if chiplet is the silver bullet to die costs, I don't get why Nvidia won't go for it. After all, Nvidia has smart ppl too. There must be some design tradeoffs that made Nvidia engineers to believe their approach is still the best in terms of perf and area. For 4090 and 4080 price, I think Nvidia strategy is: they are higher tier, with 3000s as the lower tier more affordable option. They feel they can succeed with this strategy in the current market (1 year or so) given the competitive landscape. Then they can use 4090 and 4080s to drive up the ASP and margins. Their biggest motivation is sustain the stock price.

zackmorris 4y ago
I hadn't heard the term chiplet before, thank you! When I got out of school in the late 90s, system-on-chip (SoC) was a pipe dream. It has issues because memory might have different requirements than CPU, so it's hard to put all of the components on a single die. I always thought the situation was frankly pretty silly, because I didn't want one big die anyway. I wanted a scaleable grid of chips like the 3D processor from the Terminator 2 movie. It took over 20 years, but maybe we'll have a chance to get something like that with an open source RISC-V. Maybe an array of matchhead-size CPUs at 1 GHz costing $1 each that could be scaled to hundreds or thousands of cores with content-addressable memory for automatic caching and data locality. Then it could be treated as a single processor with access to a one large shared memory space and we could drop the (contrived) distinction between CPU/GPU/TPU/etc and get back to simple desktop computing rather than dealing with the friction of vertex buffers. My guess is that Nvidia doesn't want any of that to happen, so works to maintain its dominance in single-die solutions.
- paulmd 4y ago
  "match-head sized chiplets" sort of falls into the chiplet mythos I think. Chiplets aren't magic, they actually increase power usage vs an equivalent monolithic chip, data movement is expensive and the more data you move the more expensive it is. People just think chiplets are efficient because AMD made a huge node leap (GF 12nm to TSMC 7nm is like, more than a full node, probably at least 1.5 if not 2) at the same time, but chiplets have their own costs. The smaller you split the chiplets, the more data is moving around. And the more power you'll burn. It's not desirable to go super small, you want some reasonably-sized chiplet to minimize data movement. Even if you keep the chiplets "medium-sized" and just use a lot of them... there is still some new asymptotic efficiency limit where data movement power starts to overwhelm your savings from clocking the chips lower/etc. And there's copper-copper bonding to try and fix that, but that makes thermal density even worse (and boy is Zen4 hot already... 95C under any load). Like everything else, it's just kicking the can down the road, it doesn't solve all the problems forever.
dotnet00 4y ago
It isn't the case that NVIDIA won't go for a chiplet design, they're working on it but it isn't ready yet. Current expectations for NVIDIA's chiplets to be ready seem to be for either a refresh of the 50xx series or the 60xx series.
paulmd 4y ago
Even AMD is only splitting off IO dies this generation. That's an advantage for sure, because you can push 20% or so of your die off to separate chiplets, which means you can go 25% larger than you otherwise could with a purely monolithic design. (and in particular you can also push them off to N6 or some other super-cheap node and save your super-expensive N5 allocation for the GCD... at the cost of some power for infinity fabric to send a TB/s of data around between chiplets) But so far nobody seems to have progressed on splitting the GCD itself... which has always been the actual goal here. Multiple chiplets computing together like one, transparently without software needing to be catered to like SLI. AMD's general approach seems to be "big cache-coherent interconnect between GCD chiplets", which is generally unsurprising on multiple levels (it's how infinity fabric already works in their CPUs, and it's how SMP generally works in general today almost everywhere) but there still seems to be some barrier to making it work with graphics. There are of course a lot of gotchas - like temporal data or needing access to other parts of the frame that may be in another GPU - but cache-coherent interconnects generally solve this. But yeah, NVIDIA isn't avoiding some silver bullet here, they've actually been on the forefront of this with NVSwitch since Pascal, that was the first serious "multi-GPU that acts like one" attempt. I don't know why they're not doing it, or at least the "split the IO dies off" thing. edit: thinking about this a bit more, one reason they may not be doing the MCD/io die idea is because they're doing L2 instead of L3. L3 can be pushed off to a separate die... L2 is really part of the SM, it's a lot "closer". Again, they may have engineering reasons to favor L2 over L3, or it may be a patent thing.

zackmorris 4y ago

I hadn't heard the term chiplet before, thank you! When I got out of school in the late 90s, system-on-chip (SoC) was a pipe dream. It has issues because memory might have different requirements than CPU, so it's hard to put all of the components on a single die. I always thought the situation was frankly pretty silly, because I didn't want one big die anyway. I wanted a scaleable grid of chips like the 3D processor from the Terminator 2 movie. It took over 20 years, but maybe we'll have a chance to get something like that with an open source RISC-V. Maybe an array of matchhead-size CPUs at 1 GHz costing $1 each that could be scaled to hundreds or thousands of cores with content-addressable memory for automatic caching and data locality. Then it could be treated as a single processor with access to a one large shared memory space and we could drop the (contrived) distinction between CPU/GPU/TPU/etc and get back to simple desktop computing rather than dealing with the friction of vertex buffers. My guess is that Nvidia doesn't want any of that to happen, so works to maintain its dominance in single-die solutions.

paulmd 4y ago
"match-head sized chiplets" sort of falls into the chiplet mythos I think. Chiplets aren't magic, they actually increase power usage vs an equivalent monolithic chip, data movement is expensive and the more data you move the more expensive it is. People just think chiplets are efficient because AMD made a huge node leap (GF 12nm to TSMC 7nm is like, more than a full node, probably at least 1.5 if not 2) at the same time, but chiplets have their own costs. The smaller you split the chiplets, the more data is moving around. And the more power you'll burn. It's not desirable to go super small, you want some reasonably-sized chiplet to minimize data movement. Even if you keep the chiplets "medium-sized" and just use a lot of them... there is still some new asymptotic efficiency limit where data movement power starts to overwhelm your savings from clocking the chips lower/etc. And there's copper-copper bonding to try and fix that, but that makes thermal density even worse (and boy is Zen4 hot already... 95C under any load). Like everything else, it's just kicking the can down the road, it doesn't solve all the problems forever.

dotnet00 4y ago

It isn't the case that NVIDIA won't go for a chiplet design, they're working on it but it isn't ready yet. Current expectations for NVIDIA's chiplets to be ready seem to be for either a refresh of the 50xx series or the 60xx series.

paulmd 4y ago

Even AMD is only splitting off IO dies this generation. That's an advantage for sure, because you can push 20% or so of your die off to separate chiplets, which means you can go 25% larger than you otherwise could with a purely monolithic design. (and in particular you can also push them off to N6 or some other super-cheap node and save your super-expensive N5 allocation for the GCD... at the cost of some power for infinity fabric to send a TB/s of data around between chiplets) But so far nobody seems to have progressed on splitting the GCD itself... which has always been the actual goal here. Multiple chiplets computing together like one, transparently without software needing to be catered to like SLI. AMD's general approach seems to be "big cache-coherent interconnect between GCD chiplets", which is generally unsurprising on multiple levels (it's how infinity fabric already works in their CPUs, and it's how SMP generally works in general today almost everywhere) but there still seems to be some barrier to making it work with graphics. There are of course a lot of gotchas - like temporal data or needing access to other parts of the frame that may be in another GPU - but cache-coherent interconnects generally solve this. But yeah, NVIDIA isn't avoiding some silver bullet here, they've actually been on the forefront of this with NVSwitch since Pascal, that was the first serious "multi-GPU that acts like one" attempt. I don't know why they're not doing it, or at least the "split the IO dies off" thing. edit: thinking about this a bit more, one reason they may not be doing the MCD/io die idea is because they're doing L2 instead of L3. L3 can be pushed off to a separate die... L2 is really part of the SM, it's a lot "closer". Again, they may have engineering reasons to favor L2 over L3, or it may be a patent thing.