4 ms·

The Nvidia technical brief says 208 billion transistors. https://resources.nvidia.com/en-us-blackwell-architecture https://resources.nvidia.com/en-us-blackwell

The Nvidia technical brief says 208 billion transistors. https://resources.nvidia.com/en-us-blackwell-architecture https://resources.nvidia.com/en-us-blackwell-architecture Blackwell uses the TSMC 4NP process. It has two layers. A very back of the envelope estimate: 750mm^2 / (208/2) * 10^9 = 7211 nm^2 85 nm x 85 nm NB: process feature size does not equal transistor size. Process feature size doesn't even equal process feature size.

dist-epoch 1y ago
You also need space for wires, ..., etc, right? It's not just transistors.
- CalChris 1y ago
  The wires didn't fit on the back of the envelope.
  - a_wild_dandan 1y ago
    I love this retort and I'm stealing it.
- amelius 1y ago
  The wires run over the transistors.
- gchadwick 1y ago
  The wires sit on top of the transistors. Many layers of them in a modern process. However you can't always pack the transistors as dense as you would like because you can't fit the wiring for them in above at the same density. Plus there are various 'design rules' that constrain how things get placed. These are needed to ensure manufacturing is successful and achieved good yield. An important set of rules are the 'antenna rules' that requires the insertion of antenna diodes (using silicon reducing transistor density) to prevent circuitry being destroyed during manufacturing: https://www.zerotoasiccourse.com/terminology/antenna-report/ https://www.zerotoasiccourse.com/terminology/antenna-report/
gchadwick 1y ago
> It has two layers Where did you get that from? Pretty sure it's a single planar set of transistors. Those transistors are manufactured using multiple layers of mask. FinFET transistors are described as 3D or non-planar but crucially this isn't allowing transistor on transistor stacking you've just got the gate structure of the FinFET poking out above the plane of the rest of the transistors. Silicon on silicon die stacking is a possibility but limits your power and GPUs run very hot so it's not an option for them.
- murderfs 1y ago
  GPUs are not particularly hot for compute silicon, they just have ridiculously huge dies. Comparing the 5090 to a Core Ultra 285K, the GPU has a 750 mm^2 die compared to the CPU's 243 mm^2, but has a peak power of 575W compared to 250W. The CPU uses 25% more power per area, and that's before considering the fact that consumer CPUs are packaged for user installation, so there's an extra heatspreader on top of the die, whereas GPUs are sold as integrated units, so the heatsink sits directly on top of the die.
  - kvemkon 1y ago
    > consumer CPUs are packaged for user installation I'd say advanced users or skilled staff. 20+ years ago e.g. Athlon XP had a small CPU die in the middle and 4 round spacers in the corners for a proper heatspreader installation. Despite the CPU die wouldn't clock down and go in flames in case of cooler removal during operation. Nowadays with a safer CPU monitoring its temperature, one has to risk to remove the heatspreader and replace it with "special" direct die cooling resulting in either a bit more performance or 15-20 grad lower temperatures or a smaller or a silent cooler. One is free to choose. Sure, even advanced user must take more care working around the naked die. But the technology to make this safer than before could have also matured.
    - adrian_b 1y ago
      Half a century ago, neither Pentium 4 nor Athlon would go in flames without a cooler, but because they included thermal protection circuits that stop the clock above a certain temperature threshold, which cannot be modified by users. The halting of the clock is invisible for software, except that frequent and/or long clock haltings will lengthen accordingly the execution time of a long task that has spanned over time intervals when the clock has been stopped. The latest CPUs still include such a thermal protection, which is independent of the turbo clock frequency management, which is based on the power and current limits, which can be modified by the user, e.g. for overclocking. Older CPUs, until about the middle of the nineties, could be destroyed by inadequate cooling. This has forced both Intel and AMD to implement thermal protections, to avoid requests for CPU replacements.
bgnn 1y ago
This assumes 100% utilization. Ralistically the utilization (active device area wrt total die area) 70-75% at best.

dist-epoch 1y ago

You also need space for wires, ..., etc, right? It's not just transistors.

CalChris 1y ago
The wires didn't fit on the back of the envelope.
- a_wild_dandan 1y ago
  I love this retort and I'm stealing it.
amelius 1y ago
The wires run over the transistors.
gchadwick 1y ago
The wires sit on top of the transistors. Many layers of them in a modern process. However you can't always pack the transistors as dense as you would like because you can't fit the wiring for them in above at the same density. Plus there are various 'design rules' that constrain how things get placed. These are needed to ensure manufacturing is successful and achieved good yield. An important set of rules are the 'antenna rules' that requires the insertion of antenna diodes (using silicon reducing transistor density) to prevent circuitry being destroyed during manufacturing: https://www.zerotoasiccourse.com/terminology/antenna-report/ https://www.zerotoasiccourse.com/terminology/antenna-report/

gchadwick 1y ago

> It has two layers Where did you get that from? Pretty sure it's a single planar set of transistors. Those transistors are manufactured using multiple layers of mask. FinFET transistors are described as 3D or non-planar but crucially this isn't allowing transistor on transistor stacking you've just got the gate structure of the FinFET poking out above the plane of the rest of the transistors. Silicon on silicon die stacking is a possibility but limits your power and GPUs run very hot so it's not an option for them.

murderfs 1y ago
GPUs are not particularly hot for compute silicon, they just have ridiculously huge dies. Comparing the 5090 to a Core Ultra 285K, the GPU has a 750 mm^2 die compared to the CPU's 243 mm^2, but has a peak power of 575W compared to 250W. The CPU uses 25% more power per area, and that's before considering the fact that consumer CPUs are packaged for user installation, so there's an extra heatspreader on top of the die, whereas GPUs are sold as integrated units, so the heatsink sits directly on top of the die.
- kvemkon 1y ago
  > consumer CPUs are packaged for user installation I'd say advanced users or skilled staff. 20+ years ago e.g. Athlon XP had a small CPU die in the middle and 4 round spacers in the corners for a proper heatspreader installation. Despite the CPU die wouldn't clock down and go in flames in case of cooler removal during operation. Nowadays with a safer CPU monitoring its temperature, one has to risk to remove the heatspreader and replace it with "special" direct die cooling resulting in either a bit more performance or 15-20 grad lower temperatures or a smaller or a silent cooler. One is free to choose. Sure, even advanced user must take more care working around the naked die. But the technology to make this safer than before could have also matured.
  - adrian_b 1y ago
    Half a century ago, neither Pentium 4 nor Athlon would go in flames without a cooler, but because they included thermal protection circuits that stop the clock above a certain temperature threshold, which cannot be modified by users. The halting of the clock is invisible for software, except that frequent and/or long clock haltings will lengthen accordingly the execution time of a long task that has spanned over time intervals when the clock has been stopped. The latest CPUs still include such a thermal protection, which is independent of the turbo clock frequency management, which is based on the power and current limits, which can be modified by the user, e.g. for overclocking. Older CPUs, until about the middle of the nineties, could be destroyed by inadequate cooling. This has forced both Intel and AMD to implement thermal protections, to avoid requests for CPU replacements.

bgnn 1y ago

This assumes 100% utilization. Ralistically the utilization (active device area wrt total die area) 70-75% at best.