7 ms·

The best chiplet interconnect may turn out to be no interconnect at all. Wafer scale integration [1] has come up periodically over the years. In short, just m

by retrac 5y ago

The best chiplet interconnect may turn out to be no interconnect at all. Wafer scale integration [1] has come up periodically over the years. In short, just make a physically larger integrated circuit, potentially as large as the entire wafer -- like a foot across. As I understand it, there's no particular technical hurdle, and indeed the progress with self-healing and self-testing designs with redundancy to improve yield for small processors, also makes really large designs more feasible than in the past. The economics never worked out in the favour of this approach before, but now we're at the scaling limit maybe that will change. At least one company is pursuing this at the very high end. The Cerebras WFE-2 [2] ("wafer scale engine") has 2.6 trillion transistors with 800,000 cores and 48 gigabytes of RAM, on a single, giant, integrated circuit (shown in the linked article). I'm just an interested follower of the field, no expert, so what do I know. But I think that we may see a shift in that direction eventually. Everything on-die with a really big die. System on a chip, but for the high end, not just tiny microcontrollers. [1] https://en.wikipedia.org/wiki/Wafer-scale_integration https://en.wikipedia.org/wiki/Wafer-scale_integration [2] https://www.zdnet.com/article/cerebras-continues-absolute-domination-of-high-end-compute-it-says-with-worlds-hugest-chip-two-dot-oh/ https://www.zdnet.com/article/cerebras-continues-absolute-do...

nynx 5y ago
There are some new chip manufacturing technique coming down the pipeline, which will lead to prices dropping and likely "wafer-scale" will get to the mainstream.
- tragictrash 5y ago
  Could you elaborate? Would love to know more.
  - nynx 5y ago
    Unfortunately, I cannot.
- AbbeSomething 5y ago
  Can you clarify what you mean? How can anything that would use a major share of a wafer or even a whole one be mainstream? Producing a wafer is expensive, the chips are only available to ordinary people in high income markets because you get hundreds or even thousands from a single wafer.
AceJohnny2 5y ago
To clarify and contextualize a bit what you're saying: The one big obstacle in creating larger chips is defects. There's just a statistical chance of there being a defect on any given surface area of the wafer, defect which generally breaks the chip that occupies that area of the wafer. So historically, the approach was to make more smaller chips and trash those chips on the wafer affected by defects. Then came the "chiplet" approach where they can assemble those functional chips into a larger meta-chip (like the Apple M1 Ultra). But as you're saying, changes in the way chips are designed can make them resilient to defects, so you no longer need to trash that chip on your wafer that's affected by such a defect, and can thus design a larger chip without fear of defects. (Of course such an approach requires a level of redundancy in the design, so there's a tradeoff)
- AceJohnny2 5y ago
  I'll add that many DRAM chips already do something like this, but ironically enough the re-routing mechanism adds complexity which is itself a source of problems, (be it manufacturing or design, such as broken timing promises) Also, NAND Flash storage (SSD) is designed around the very concept of re-routing around bad blocks, because the very technology means they have a wear-life.
  - Dylan16807 5y ago
    > I'll add that many DRAM chips already do something like this, but ironically enough the re-routing mechanism adds complexity which is itself a source of problems, (be it manufacturing or design, such as broken timing promises) The best-performing solution there is probably software. Tell the OS about bad blocks and keep the hardware simple.
    - nine_k 5y ago
      I think this is already implemented both in Linux and in Windows; you can tell the OS which RAM ranges are defective. Doing this from the chip side is not there yet, apparently. I wonder when will this be included in the DRAM feature list, if ever. I suspect that detecting defects from the RAM side is not trivial.
kwhitefoot 5y ago
Nothing new under the sun. Ivor Catt was proposing wafer scale computing in the '70s. Large numbers of processors with the ability to route around defective units. https://www.ivorcatt.org/icrns86jun_0004.htm https://www.ivorcatt.org/icrns86jun_0004.htm
wmf 5y ago
Calling wafer-scale "no interconnect" is kind of misleading since it's still very difficult to stitch reticles and it has yield challenges.
- jacquesm 5y ago
  Indeed, that Zdnet article hints at how they solved that but doesn't provide detail enough to make sense of it.
galaxyLogic 5y ago
Sounds like a great development if it works out. But consider also that you can stick chiplets on top of each other vertically. That means you can put chiplets much closer together than if they were constrained to exist on the same single plane of the wafer. Now how about stacking wafers on top of wafers? That could be super, but there might be technical difficulties, which maybe sooner or later can be overcome.
- AceJohnny2 5y ago
  > But consider also that you can stick chiplets on top of each other vertically. The problem there is heat dissipation. Already the performance constraint on consumer chips like the Apple M1 is how well it can dissipate heat in the product it's placed in (see Macbook Air vs Mac Mini). Stacking the chips just makes it worse.
  - Iwan-Zotow 5y ago
    the problem is signal propagation for light to cross 1 feet should take ca 1ns
    - paulmd 5y ago
      3D circuits would be denser (shorter propagation distances) than a planar circuit. In fact "computronium" is sort of an idea about how dense you can conceptually make computation. You just can't really cool it that well with current technologies. Microfluidics are the current magic wand that everyone wishes existed but it's a ways away yet.
    - adwn 5y ago
      Signal propagation latency is only a problem if both end points need to be fully synchronous, which they don't have to be if they're independent units. Even the cores within single-chip multi-core CPUs aren't synchronous to each other, as far as I know.
  - GeekyBear 5y ago
    The fact that the M1 Macbook Air operates without needing a fan is very unusual for that level of performance.
AtlasBarfed 5y ago
... that's the exact opposite of every economic and yield advantage that chiplet design addresses, isn't it? Want to fine tune your chip offering to some multiple of 8 cores (arbitrary example of the # cores on the chiplet)? Just a packaging issue. Want to upbin very large corecounts that generally overclock quite well? For a massive unichip described, maybe there are sections of the chip that are clocking well and sections that aren't: you're stuck. With chiplets, you have better binning granularity and packaging. Want to fine-tune various cache levels? I believe from what I've read that AMD is doing L3 on a separate chiplet (and vertically stacking it!). So you can custom-tune the cache size for chip designs, You can custom-process different parts of the "CPU" with different processes and fabrication, possibly even different fab vendors. You can upgrade various things like memory support and other evolving things in an isolated package, which should help design and testing. The interconnects are the main problem. But then again, I can't imaging what a foot-wide CPU introduces for intra-chip communication, it probably would have it's own pseudo-interconnect highway anyway. Maybe you don't even need to reengineer some chiplets between processor generations. If the BFD of your new release is some improvement to the higher or lower cpus in the High-Low designs, but the other is the same, then that should be more organizational efficiency. Intel and others have effectively moved from gigantic integrated circuits decades ago: motherboard chipsets were always done with a separate cheaper fab that was a gen or two behind the CPU. Maybe when process tech has finally stabilized for a generation now that process technology seems to be stagnating more, then massive wafer designs will start to edge out chiplet designs, but right now it appears that the opposite has happened and will continue for the foreseeable future.
- paulmd 5y ago
  I said it upthread but maybe it's better to use the conceptual model of "chiplets but you don't cut the chiplets apart". You have some network of "chiplets" connected with some kind of interconnect (a mesh is the model everyone's used so far). So a "chiplet" (a module) could still have some of its cores turned off, or be disabled entirely, and then you just route around them. The yields are thus the exact same you would expect from taking a wafer, printing a bunch of chiplets, and then not cutting it apart. Most of them (Zen3 is >90% with 8 functional cores now iirc) will be fine. A few will fail clock bins. A few will be dead. Etc. https://cerebras.net/blog/wafer-scale-processors-the-time-has-come/ https://cerebras.net/blog/wafer-scale-processors-the-time-ha... A lot of the points you raise are fair, some of them can be engineered around and others not really. * There is no reason that different "chiplets" can't have different clocks. big.LITTLE already deals with different clock domains and rates, your interconnect should be asynchronous and then it's just an aspect of the design. Maybe it's not desirable from a programming perspective but it's not a problem in hardware, and if you don't like an uneven canvas, then just disable those modules. * However you can't do different voltages as easily. Having a FIVR (Fully-Integrated Voltage Regulator) in each "chiplet" is an option, just feed some high voltage like 2v or 3v and have the chip step it down. This is highly desirable for efficiency anyway. Or just feed a reasonably high voltage and if some of the chips don't clock well, then oh well, you use a little more voltage than necessary (Intel does this for Alder Lake - the chip uses the greater of "V_big" and "V_little" for whatever values are needed for frequencies at the moment - if the little cores get more voltage than they need, oh well). I don't think clock variations/chip quality generally varies as much as people imagine it does, especially once a process gets relatively mature, you can come up with some relatively good bars that almost all cores will pass. * Wafer-scale chips have to be fabbed with everything on one process, that's true. With "tiles" and EMIB, or MCM, or regular stacking, you can mix and match them, including between vendors. A lot of these "mix and match" bits don't really make sense/don't really fit the release model/don't really fit the physical engineering: * A core block is already a self-contained (relatively) module that can be designed/tested/updated in isolation. Once you've got the bugs hammered out, scaling it arbitrarily large isn't a problem, and again, you probably are organizing it into "modules" of 8 cores or so anyway for organization's sake, even with wafer-scale. * "version friction" is already a thing even in other chips. If RDNA2 isn't ready to go when AMD designs their Zen3 APUs... oh well, it doesn't go until the next generation. You could certainly iterate versions more quickly, but you can't change these behind your customer's back - if RDNA2 becomes available at some later date, you can't just swap it in, that's a new product and has to be validated and designed for separately. * "steppings" already handle this in traditional chips. If there is some minor improvement in the memory controller, it becomes a new stepping of the chip. If it's a big change... it's a new release. The big advantage of wafer-scale over current MCM/interposer tech at an engineering level is the much lower cost of data movement. MCM/interposer is expensive to push a signal all the way through a big interposer somewhere else, and it takes quite a lot of cache to even attempt to hide this. "Active interposer", EMIB, and advanced bonding technologies can help reduce this a lot, but it's "cheaper" to just push it to the module next to you and then have it pass the message along in turn. MCM also really doesn't go to the scale wafer-scale does. What if I want a package with 64 CCDs on it? That's really what wafer-scale is about. But you can't do that with MCM since that's bigger than the reticle limit used to print an interposer. What you'd have to do is print "bridges" that go under the edges of the chiplets and carry the signal, and then mount the whole thing on an inert substrate (not printed like an interposer). And... that's exactly what EMIB does. But you'd have to do a ton of packaging work to get the size of chip that wafer-scale can gives you, each of those packaging stages (attaching each bridge, etc) has failure rates too. And from the sounds of it, it's actually quite high (AMD is supposedly taking a big hit on Milan-X yields due to the cache packaging step).
Iwan-Zotow 5y ago
speed of light ~ 1*10^9 feet/sec To cross one foot - no less than 10^-9 sec = 1ns
FredPret 5y ago
Instead of microcircuits, megacircuits. I like it
kurthr 5y ago
The typical problem is the ~30mm retical dimension limit. Unless you want to have the same pattern copied ~100x over the wafer you'll need to have more unique masks... A LOT of masks, and many more mask steps (because they can't step the same mask). For reasonable (finfet) geometries that is much too expensive, even if you could buy the required aligners. I suspect stacked wafer is much more cost competitive, and you get to use appropriate (RAM, CMOS, Flash) processes for each layerso long as their sizes match. Via size and density are such that speed is less of an issue than going to package and the yield hit isn't more than 2-3x.
jpmattia 5y ago
> Wafer scale integration [1] has come up periodically over the years. Not merely years, but decades. I worked with a wafer-scale group in MIT Lincoln Laboratory back in the late 80s. I'd say there is a reason the technology hasn't taken off in the past 35 years, but who knows, maybe now is the time for wafer scale integration to shine. Incidentally, that group was originally Ken Olsen's group, after he left to go found Digital Equipment Corporation (DEC). They had a lot of talent and produced some interesting wafers. The tech back then (4in wafers iirc) used laser reconfigurability to route around bad cells.
- jacquesm 5y ago
  I think a part of this is that the clock speed increases made most other more expensive solutions not yet commercially viable, but now that 'the end' is in sight a lot of those more expensive solutions are making a come back. You could see the same happening in software to mirror the far more exotic hardware that we use now compared to single core CPUs which for a very long time were the norm.
mbrodersen 5y ago
The problem is cost. Instead of getting N CPU’s per wafer you are now getting just 1 mega-CPU. So you need to sell the mega-CPU at N times the price to get the same ROI. So a solution for super computer applications. Not for everybody else.

by retrac 5y ago

nynx 5y ago

There are some new chip manufacturing technique coming down the pipeline, which will lead to prices dropping and likely "wafer-scale" will get to the mainstream.

tragictrash 5y ago
Could you elaborate? Would love to know more.
- nynx 5y ago
  Unfortunately, I cannot.
AbbeSomething 5y ago
Can you clarify what you mean? How can anything that would use a major share of a wafer or even a whole one be mainstream? Producing a wafer is expensive, the chips are only available to ordinary people in high income markets because you get hundreds or even thousands from a single wafer.

AceJohnny2 5y ago

To clarify and contextualize a bit what you're saying: The one big obstacle in creating larger chips is defects. There's just a statistical chance of there being a defect on any given surface area of the wafer, defect which generally breaks the chip that occupies that area of the wafer. So historically, the approach was to make more smaller chips and trash those chips on the wafer affected by defects. Then came the "chiplet" approach where they can assemble those functional chips into a larger meta-chip (like the Apple M1 Ultra). But as you're saying, changes in the way chips are designed can make them resilient to defects, so you no longer need to trash that chip on your wafer that's affected by such a defect, and can thus design a larger chip without fear of defects. (Of course such an approach requires a level of redundancy in the design, so there's a tradeoff)

AceJohnny2 5y ago
I'll add that many DRAM chips already do something like this, but ironically enough the re-routing mechanism adds complexity which is itself a source of problems, (be it manufacturing or design, such as broken timing promises) Also, NAND Flash storage (SSD) is designed around the very concept of re-routing around bad blocks, because the very technology means they have a wear-life.
- Dylan16807 5y ago
  > I'll add that many DRAM chips already do something like this, but ironically enough the re-routing mechanism adds complexity which is itself a source of problems, (be it manufacturing or design, such as broken timing promises) The best-performing solution there is probably software. Tell the OS about bad blocks and keep the hardware simple.
  - nine_k 5y ago
    I think this is already implemented both in Linux and in Windows; you can tell the OS which RAM ranges are defective. Doing this from the chip side is not there yet, apparently. I wonder when will this be included in the DRAM feature list, if ever. I suspect that detecting defects from the RAM side is not trivial.

kwhitefoot 5y ago

Nothing new under the sun. Ivor Catt was proposing wafer scale computing in the '70s. Large numbers of processors with the ability to route around defective units. https://www.ivorcatt.org/icrns86jun_0004.htm https://www.ivorcatt.org/icrns86jun_0004.htm

wmf 5y ago

Calling wafer-scale "no interconnect" is kind of misleading since it's still very difficult to stitch reticles and it has yield challenges.

jacquesm 5y ago
Indeed, that Zdnet article hints at how they solved that but doesn't provide detail enough to make sense of it.

galaxyLogic 5y ago

Sounds like a great development if it works out. But consider also that you can stick chiplets on top of each other vertically. That means you can put chiplets much closer together than if they were constrained to exist on the same single plane of the wafer. Now how about stacking wafers on top of wafers? That could be super, but there might be technical difficulties, which maybe sooner or later can be overcome.

AceJohnny2 5y ago
> But consider also that you can stick chiplets on top of each other vertically. The problem there is heat dissipation. Already the performance constraint on consumer chips like the Apple M1 is how well it can dissipate heat in the product it's placed in (see Macbook Air vs Mac Mini). Stacking the chips just makes it worse.
- Iwan-Zotow 5y ago
  the problem is signal propagation for light to cross 1 feet should take ca 1ns
  - paulmd 5y ago
    3D circuits would be denser (shorter propagation distances) than a planar circuit. In fact "computronium" is sort of an idea about how dense you can conceptually make computation. You just can't really cool it that well with current technologies. Microfluidics are the current magic wand that everyone wishes existed but it's a ways away yet.
  - adwn 5y ago
    Signal propagation latency is only a problem if both end points need to be fully synchronous, which they don't have to be if they're independent units. Even the cores within single-chip multi-core CPUs aren't synchronous to each other, as far as I know.
- GeekyBear 5y ago
  The fact that the M1 Macbook Air operates without needing a fan is very unusual for that level of performance.

AtlasBarfed 5y ago

... that's the exact opposite of every economic and yield advantage that chiplet design addresses, isn't it? Want to fine tune your chip offering to some multiple of 8 cores (arbitrary example of the # cores on the chiplet)? Just a packaging issue. Want to upbin very large corecounts that generally overclock quite well? For a massive unichip described, maybe there are sections of the chip that are clocking well and sections that aren't: you're stuck. With chiplets, you have better binning granularity and packaging. Want to fine-tune various cache levels? I believe from what I've read that AMD is doing L3 on a separate chiplet (and vertically stacking it!). So you can custom-tune the cache size for chip designs, You can custom-process different parts of the "CPU" with different processes and fabrication, possibly even different fab vendors. You can upgrade various things like memory support and other evolving things in an isolated package, which should help design and testing. The interconnects are the main problem. But then again, I can't imaging what a foot-wide CPU introduces for intra-chip communication, it probably would have it's own pseudo-interconnect highway anyway. Maybe you don't even need to reengineer some chiplets between processor generations. If the BFD of your new release is some improvement to the higher or lower cpus in the High-Low designs, but the other is the same, then that should be more organizational efficiency. Intel and others have effectively moved from gigantic integrated circuits decades ago: motherboard chipsets were always done with a separate cheaper fab that was a gen or two behind the CPU. Maybe when process tech has finally stabilized for a generation now that process technology seems to be stagnating more, then massive wafer designs will start to edge out chiplet designs, but right now it appears that the opposite has happened and will continue for the foreseeable future.

paulmd 5y ago
I said it upthread but maybe it's better to use the conceptual model of "chiplets but you don't cut the chiplets apart". You have some network of "chiplets" connected with some kind of interconnect (a mesh is the model everyone's used so far). So a "chiplet" (a module) could still have some of its cores turned off, or be disabled entirely, and then you just route around them. The yields are thus the exact same you would expect from taking a wafer, printing a bunch of chiplets, and then not cutting it apart. Most of them (Zen3 is >90% with 8 functional cores now iirc) will be fine. A few will fail clock bins. A few will be dead. Etc. https://cerebras.net/blog/wafer-scale-processors-the-time-has-come/ https://cerebras.net/blog/wafer-scale-processors-the-time-ha... A lot of the points you raise are fair, some of them can be engineered around and others not really. * There is no reason that different "chiplets" can't have different clocks. big.LITTLE already deals with different clock domains and rates, your interconnect should be asynchronous and then it's just an aspect of the design. Maybe it's not desirable from a programming perspective but it's not a problem in hardware, and if you don't like an uneven canvas, then just disable those modules. * However you can't do different voltages as easily. Having a FIVR (Fully-Integrated Voltage Regulator) in each "chiplet" is an option, just feed some high voltage like 2v or 3v and have the chip step it down. This is highly desirable for efficiency anyway. Or just feed a reasonably high voltage and if some of the chips don't clock well, then oh well, you use a little more voltage than necessary (Intel does this for Alder Lake - the chip uses the greater of "V_big" and "V_little" for whatever values are needed for frequencies at the moment - if the little cores get more voltage than they need, oh well). I don't think clock variations/chip quality generally varies as much as people imagine it does, especially once a process gets relatively mature, you can come up with some relatively good bars that almost all cores will pass. * Wafer-scale chips have to be fabbed with everything on one process, that's true. With "tiles" and EMIB, or MCM, or regular stacking, you can mix and match them, including between vendors. A lot of these "mix and match" bits don't really make sense/don't really fit the release model/don't really fit the physical engineering: * A core block is already a self-contained (relatively) module that can be designed/tested/updated in isolation. Once you've got the bugs hammered out, scaling it arbitrarily large isn't a problem, and again, you probably are organizing it into "modules" of 8 cores or so anyway for organization's sake, even with wafer-scale. * "version friction" is already a thing even in other chips. If RDNA2 isn't ready to go when AMD designs their Zen3 APUs... oh well, it doesn't go until the next generation. You could certainly iterate versions more quickly, but you can't change these behind your customer's back - if RDNA2 becomes available at some later date, you can't just swap it in, that's a new product and has to be validated and designed for separately. * "steppings" already handle this in traditional chips. If there is some minor improvement in the memory controller, it becomes a new stepping of the chip. If it's a big change... it's a new release. The big advantage of wafer-scale over current MCM/interposer tech at an engineering level is the much lower cost of data movement. MCM/interposer is expensive to push a signal all the way through a big interposer somewhere else, and it takes quite a lot of cache to even attempt to hide this. "Active interposer", EMIB, and advanced bonding technologies can help reduce this a lot, but it's "cheaper" to just push it to the module next to you and then have it pass the message along in turn. MCM also really doesn't go to the scale wafer-scale does. What if I want a package with 64 CCDs on it? That's really what wafer-scale is about. But you can't do that with MCM since that's bigger than the reticle limit used to print an interposer. What you'd have to do is print "bridges" that go under the edges of the chiplets and carry the signal, and then mount the whole thing on an inert substrate (not printed like an interposer). And... that's exactly what EMIB does. But you'd have to do a ton of packaging work to get the size of chip that wafer-scale can gives you, each of those packaging stages (attaching each bridge, etc) has failure rates too. And from the sounds of it, it's actually quite high (AMD is supposedly taking a big hit on Milan-X yields due to the cache packaging step).

Iwan-Zotow 5y ago

speed of light ~ 1*10^9 feet/sec To cross one foot - no less than 10^-9 sec = 1ns

FredPret 5y ago

Instead of microcircuits, megacircuits. I like it

kurthr 5y ago

The typical problem is the ~30mm retical dimension limit. Unless you want to have the same pattern copied ~100x over the wafer you'll need to have more unique masks... A LOT of masks, and many more mask steps (because they can't step the same mask). For reasonable (finfet) geometries that is much too expensive, even if you could buy the required aligners. I suspect stacked wafer is much more cost competitive, and you get to use appropriate (RAM, CMOS, Flash) processes for each layerso long as their sizes match. Via size and density are such that speed is less of an issue than going to package and the yield hit isn't more than 2-3x.

jpmattia 5y ago

> Wafer scale integration [1] has come up periodically over the years. Not merely years, but decades. I worked with a wafer-scale group in MIT Lincoln Laboratory back in the late 80s. I'd say there is a reason the technology hasn't taken off in the past 35 years, but who knows, maybe now is the time for wafer scale integration to shine. Incidentally, that group was originally Ken Olsen's group, after he left to go found Digital Equipment Corporation (DEC). They had a lot of talent and produced some interesting wafers. The tech back then (4in wafers iirc) used laser reconfigurability to route around bad cells.

jacquesm 5y ago
I think a part of this is that the clock speed increases made most other more expensive solutions not yet commercially viable, but now that 'the end' is in sight a lot of those more expensive solutions are making a come back. You could see the same happening in software to mirror the far more exotic hardware that we use now compared to single core CPUs which for a very long time were the norm.

mbrodersen 5y ago

The problem is cost. Instead of getting N CPU’s per wafer you are now getting just 1 mega-CPU. So you need to sell the mega-CPU at N times the price to get the same ROI. So a solution for super computer applications. Not for everybody else.