8 ms·
Chinese researchers planning 1,600-core chips that use an entire wafer
- rightbyte 3y agoThe concept is interesting. I guess it will have to be able to route around broken cores?
- dragontamer 3y agoI've seen other projects that market themselves as "wafer scale". https://www.cerebras.net/product-chip/ https://www.cerebras.net/product-chip/ > I guess it will have to be able to route around broken cores? Yeah, but you'll also have to route around broken routes, and that starts to get a bit too much chicken-and-egg problem for me. I guess you could design something akin to error correction codes, meaning you're resilient to X failures. Ex: a 64-bit bus could have physically Single Error Correction, Double Error Detection, which IIRC would be 72 physical wires. That means any wire can completely fail, but you still have a 64-bit bus (indeed, the 8x error-correction wires could all fail and you'd still have a 64-bit bus). ------------ At some point, it makes more sense to cut the chips out, test them for reliability. Then cut the router out, and test those for reliability, and then finally glue them together. On the other hand, doing it all on one wafer has cost savings / manufacturing simplicity. The math is likely difficult for optimizing over costs, production speeds, and so forth.
- fspeech 3y agoYou can design supervisory nodes and routing elements with relaxed design rules that guarantee high yield for these critical elements.
- shreezus 3y agoThere are techniques to disable nodes that fail testing, so it shouldn't be a problem (within reason).
- a-french-anon 3y ago1.7%
- Alifatisk 3y agoWhat do you mean?
- a-french-anon 3y agohttps://www.semiaccurate.com/2009/09/15/nvidia-gt300-yeilds-under-2/ https://www.semiaccurate.com/2009/09/15/nvidia-gt300-yeilds-... Old memes die hard
- ethbr1 3y agoDoes anyone know how yields are billed, e.g. by TSMC? If I'm Nvidia, and I contract for X volume on Y process, and TSMC delivers it with Z yield... how does the +/- to Z work? I'd assume it isn't completely Nvidia's to eat? More like there's an expected yield and then bonuses / penalties to TSMC for above/below that?
- andrewstuart 3y agoI would have said 1.3.
- WhereIsTheTruth 3y agoThose who try may get 1.7% Those who don't may forever get 0%
- adhesive_wombat 3y agoWayne Gretzky's post-retirement career in semiconductor engineering may not have been expected, but seems to be going well.
- blihp 3y agoThat's because he doesn't design for where the state of the art chips are, he designs for where they are going to be.
- ramshanker 3y agoNow that Cerebrus has proven it works, I would love to have an x86 / ARM / NVidia do this. And for best results, onboard one of the memory maker as well. Cerebrus seems to have underestimated memory requirement of LLM. So imagine, 16 H200 GPU along with single digit TB HBM memory stitched together on a single substrate wafer. It seems doable with correct technology. Go for it China. You are in good track here.
- verall 3y ago> 16 H200 GPU along with single digit TB HBM memory stitched together on a single substrate wafer How on earth would you cool this?
- MacsHeadroom 3y agoSubmersion cooling.
- aleph_minus_one 3y ago> How on earth would you cool this? With liquid nitrogen. :-)
- snakeyjake 3y agoSeymour Cray solved this problem 40 years ago. https://en.wikipedia.org/wiki/Cray-2#/media/File:Cray2.jpg https://en.wikipedia.org/wiki/Cray-2#/media/File:Cray2.jpg Bring back toxic waterfalls.
- shrx 3y agoThe issue with Fluorinert is not toxicity but a very high global warming potential.
- mensetmanusman 3y agoFluorinerts with very low global warming potentials have been invented since then.
- ethbr1 3y agoThat sounds great... until you get a defect on your networking block. Then what? You've got a heterogenous network with tons of "this core to this core is not like the others" exceptions (latency, bandwidth, etc). I know chip-to-chip/memory interconnects burn a ton of power, but fabbing discrete "biggest chip we can get with decent yield" still seems a solid tradeoff in the reality of < 100% yields. Does anyone have a link or search phrases on how this is currently handled for high-chiplet counts? E.g. interconnection routing architectures that are still reasonable with random manufacturing-time failing links
- arcticbull 3y agoI assume it's probably an 1800 core wafer and they just cut fuse off the 200 defective cores. Some redundancy is likely just built-in based on the expectations of process reliability. Probably multiple networking blocks, too, and you'd use less demanding process features on the things that can't be duplicated. In fact you could probably even have FPGA-style soft programmable fabric interconnects to work around process failures.
- ethbr1 3y agoThat's great if it's just hitting compute cores. But when your wafer networking flows through cores (because it's cores-all-the-way-down), a defective core starts to impact network performance. Which cascades into cache, memory, locality, etc. Which starts to make a very unpredictable hardware system for software to reason about. See also dragontamer's comment down below.
- fspeech 3y agoNot all transistors have to yield the same. You can use more forgiving design rules to ensure that critical network elements don't fail. There is also post fabrication repair. It all comes down to economics. There is no fundamental problem engineering wise.
- adgjlsfhk1 3y ago
- lebean 3y agoThey can use these for their high speed trains!
- deleted 3y ago[deleted]
- kylehotchkiss 3y agoWhat about trains requires this level of processing power???
- hulitu 3y ago> China planning 1600-core chips that use an entire wafer – 'wafer-scale' designs ... and they will cool it by pouring water on it. /s
- throwup238 3y agoPretty much. This is the way Cerebrus does it: https://web.archive.org/web/20230812020202/https://www.youtube.com/watch?v=pzyZpauU3Ig https://web.archive.org/web/20230812020202/https://www.youtu...
- cm2187 3y agoThe article talks about chiplets. I presume the wafer will still be cut into distinct chips? I thought there were thermal (and yield) reasons to not making chips that are too large.
- deleted 3y ago[deleted]
- blihp 3y agoMy take was that they currently have a working chiplet design and are looking to move to a wafer-scale (i.e. not cut up) design. Thermal/power/yield are all issues and the design has to take all of that into account. Cerebras has done it for their NN processors, so it has been done before.
- dboreham 3y agoThe article doesn't note this, but wafer scale integration is a very old idea. We discussed it at Inmos back in the day, since often the systems we built essentially consisted of many CPU die sliced out of the wafer, bonded into a package, then tiled onto a PCB[1]. But there are...issues: cooling for one. Iann Barron joked that you could make a toaster from two WSI wafers running full-tilt. [1] https://twitter.com/tnmoc/status/429638751904878592 https://twitter.com/tnmoc/status/429638751904878592
- nickpsecurity 3y agoThere’s been quite a few. I can’t remember the first one I found which was also heavily analog. Here’s another: https://www.kip.uni-heidelberg.de/vision/previous-projects/facets/neuromorphic-hardware/waferscale-integration-system/hicann/ https://www.kip.uni-heidelberg.de/vision/previous-projects/f... https://iopscience.iop.org/article/10.1088/2634-4386/acf7e4 https://iopscience.iop.org/article/10.1088/2634-4386/acf7e4
- buildbot 3y agoThe HICANN chip is really interesting, never seen or heard of it before!
- nerpderp82 3y agoThat is super cool. I wrote to SGS-Thomson when I was in highschool and they sent me what felt like a refrigerator box of manuals for the 400 through the 9000. Huge transputer fan. The simplicity was striking, and that you could construct a system that could handle huge numbers of threads communicating across the network with almost no "system software" in the way was mind blowing. Most famously, Gene Amdahl started https://en.wikipedia.org/wiki/Trilogy_Systems https://en.wikipedia.org/wiki/Trilogy_Systems in the 80s to explore this idea. The calculus changes when you don't have to dice the wafer, packaging, etc. I'd say that we clock things now at the highest speed that we can safely remove heat, so these wafer scale chips, we trade frequency for area and need/should clock them much slower. At large production runs, the wafer in a Cerebras is 20k each for a system costing millions and the primary engineering feat is still cooling. I'd love to see a WSI system utilizing NTV (near threshold voltage) logic. https://semiengineering.com/near-threshold-computing-2/ https://semiengineering.com/near-threshold-computing-2/ Another interesting design pattern that has arisen is that Cerebras, Esperanto, Tenstorrent, and InspireSemi are all mesh networks using message passing. What kinds of things did you work on at Inmos?
- TriangleEdge 3y agoWhat kind of wattage is expected for this? And how is heat management done?
- jeroen79 3y agoHow the hell are they gonna cool a cpu like that?
- sandworm101 3y agoBig block of copper. Water pumped through channels. The total heat output isnt all that big comparred to other water-cooled processes. A car engine cylinder pumps out more heat across a similar area. You will just need pumps and fans bigger than the toy parts used in normal pc cooling.
- hyperthesis 3y agoYou just need a car radiator and engine.
- sandworm101 3y agoNot really. A small automotive radiator, like on a motorcycle, can easily handle a few thousand watts. You just need a 200ish watt fan and a 100+ watt pump, not the sort of thing for home use but nothing extreme in the world of cooling products.
- Torkel 3y ago"China" is "planning" to do this...? A better title might be: "Researchers in China studying 1600-core chip".
- dang 3y agoOk, we've replaced China with some Chinese researchers in the title above.
- brindlejim 3y agoSoftware determines whether good hardware succeeds or fails. China has yet to build successful software ecosystems on top of its hardware innovations.
- Barrin92 3y agoWait what, China has by significant margin the second largest software ecosystem in the world, and in VC terms is comparable to the US.
- brindlejim 3y agoChina runs on Intel chips like everyone else. It's AI researchers beg for Nvidia chips like everyone else. And its mobile companies use Android. Chips that break away from those standards have little chance of success.
- foofie 3y ago> China has yet to build successful software ecosystems on top of its hardware innovations. I'm not sure your belief is grounded in reality. I'd go as far as to assert that if China was able to research and develop these chips, both their design and production processes, they certainly are not leaving software as an afterthought. Nevertheless, even entertaining your fantasy, once these chips are out and people like you and me are able to take these toys out to play with them, you'll soon get software that does something interesting and useful. Software is hardly the hard part, or even costlier.
- Wytwwww 3y ago> Software is hardly the hard part, or even costlier Tell that to Nvidia. HW is a commodity with relatively low margins unless you can lock in your users in some other way.
- foofie 3y ago> Tell that to Nvidia. Nvidia is an excellent example. Without the hardware part, they would simply not have a product line. As they developed expertise in hardware design and production, they are now one of the most valuable companies in existence. Some market segments even spend thousands of dollars in Nvidia's hardware without having any expectation or plan to use any of NVidia's drivers. Nvidia proves hardware is the hard part.
- leashless 3y agoBeen hearing it for 30 years. Finally?
- femto 3y agoThere were experiments with wafer scale FPGAs in the 1990s. The idea was that being programmable, the final chip could be programmed to route around defects. Lasers were also used to eliminated defective cells.
- FpUser 3y ago>"The latter has only been managed by Cerebras so far, but it looks like Chinese developers are looking towards them as well." Cerebra's wafer has 850,000 cores which totally dwarves 1600 cores on Chinese wafer. I did read though that Cerebra cores optimized for tensor ops. Does Chinese version have more universal cores or it just way smaller clone of Cerebra?