8 ms·
Cerebras Wafer Scale Engine Gen2 7nm 2.6T Transistors
- segfaultbuserr 6y agoHow do you connect such an enormous chip to a circuit board? What packaging and wire bonding technologies are available? I don't think it uses a conventional package like a (Flip-Chip) BGA - the process of soldering a 65535-pin BGA, and instantly losing millions of dollars if there are defective joints is simply unimaginable. The huge chip also have a heat dissipation problem that conventional packaging is unable to solve.
- aveni 6y agoA lot of system-related questions were answered at HotChips last year: https://www.hotchips.org/hc31/HC31_1.13_Cerebras.SeanLie.v02.pdf https://www.hotchips.org/hc31/HC31_1.13_Cerebras.SeanLie.v02... And SC19: https://secureservercdn.net/198.12.145.239/a7b.fcb.myftpupload.com/wp-content/uploads/2019/11/Cerebras_for_SC19.pdf https://secureservercdn.net/198.12.145.239/a7b.fcb.myftpuplo...
- segfaultbuserr 6y agoThanks for the links! So everything is custom-developed, including wire bonding, packaging, connectors, heat spreaders, an impressive achievement. Unfortunately it's only a three-page overview, I'd love to read more on technical details.
- npunt 6y agoIt's mind-boggling to consider what it takes to power and cool a 15 kilowatt 'chip'. That is a huge amount of electricity.
- bleepblorp 6y agoFor comparison's sake, 15kw is enough power to heat a house in a moderate winter climate. Getting a furnance worth of power into and out of a chip without melting it is, indeed, very impressive.
- segfaultbuserr 6y ago15 kW is 7x more than anything that can be (legally) powered from a standard European 230 V, 10 A outlet. With a 2.3 kW wall outlet, you can already power a Bitcoin miner unit or a small IBM mainframe computer. It's indeed impressive. Which brings us to the next topic: What does the power supply look like, and how does power delivery work? Given the huge amount of power, the PSU for the chip alone is an interesting piece of electronics. Assuming its power supply is similar to a PC or server... First, the mains power is converted to a system DC voltage such as 12 V. For a typical modern cheap and boring switched-mode power supply, the standard efficiency requirement is 80%. But in this case, it means the PSU itself will waste 3 kW of energy, this is unacceptable both from an environmental and thermal perspective. A really good supply can achieve an efficiency of 95%-98%, and this is when things get interesting, at least you'll see some expensive controller chips and high-power, high-frequency transistors in such a PSU, terms like SiC, GaN or IGBT come to mind, switching 1,250 amps on and off. And it doesn't end here, the next step is converting the 12 V system voltage to the chip's core voltage, say 1.0 V, by a Voltage Regulator Module near or on the motherboard. Now, don't even mention the technical challenge of designing a VRM. I can't even imagine how they run the huge current on the motherboard, it's 15,000 amps! Unfortunately, there's no information about power supply and power delivery.
- avianes 6y agoThere are probably several power domains in the chip and therefore several separate power supplies, which avoids the need to manage 15kW in a single spot. Edit: In the slides shared above [1] (page 16), it says "12x 4kW hot-swappable universal PSUs". [1] : https://secureservercdn.net/198.12.145.239/a7b.fcb.myftpupload.com/wp-content/uploads/2019/11/Cerebras_for_SC19.pdf#page=16 https://secureservercdn.net/198.12.145.239/a7b.fcb.myftpuplo...
- ac29 6y agoNot sure why this was downvoted - a relatively small natural gas forced air heater as is common nearly everywhere I've lived in the US is about 50,000 BTU or very nearly 15kW. Its a lot of heat.
- ChuckMcM 6y agoDefinitely a challenge, heat dissipation is "easier" if you can glue it to a plumbed 1/2" thick copper heat exchanger. I had an opportunity see a laser sintered copper heat exchanger that was very very efficient. Pumping in water and pulling out steam efficient. It was pretty cool.
- ramshanker 6y agoI can't help imagining INTEL / AMD / NVIDIA doing the same with their own chips. Top500 Supercomputer in a single Rack. Just imagine. And if we take liberty to dream extreme, "Available on Amazon, On Demand". :D
- hinkley 6y agoSomeone is going to figure out that if they cut the wafers on a diagonal they can make even bigger wafers. And someone will be spending a lot of time trying to figure out how to support a wafer cut lengthwise without shattering it during processing.
- Tuna-Fish 6y agoThat wouldn't work. Modern chips need to be built on epitaxial surfaces, and the only plane they can reliably produce an epitaxial surface on is the horizontal one.
- evancox100 6y agoStill wondering how they’ve handled interconnecting beyond each reticle limit/exposure. Are they staggering the exposures for each layer? And how do they handle packaging/IO/power delivery? I guess these are the secret sauces and we are unlikely to get in depth information.
- gary_0 6y agoIn the PDF[0] that aveni linked above, they mention they "add wires across scribe line in partnership with TSMC, Extend 2D mesh across die". So the wafer isn't exactly one big chip. [0] https://www.hotchips.org/hc31/HC31_1.13_Cerebras.SeanLie.v02.pdf https://www.hotchips.org/hc31/HC31_1.13_Cerebras.SeanLie.v02...
- wmf 6y agoI think reticle stitching has been done previously in image sensors and silicon interposers so it may be an evolution of that.
- jszymborski 6y agoSo, just wondering from a practical perspective, how do you train/serve models from these? I doubt they play nice with e.g. PyTorch or TF. Maybe they're meant to be used with a DL compiler like TVM [0]? [0] https://tvm.apache.org/ https://tvm.apache.org/
- aveni 6y agoYou can use standard ML frameworks :) The code then goes through the Cerebras Graph Compiler to produce an executable that runs on the wafer. More of the software stack was described at HotChips today, covered by AnandTech: https://www.anandtech.com/show/16006/hot-chips-2020-live-blog-cerebras-wse-programming-300pm-pt https://www.anandtech.com/show/16006/hot-chips-2020-live-blo...
- genr8 6y agoIronic this article was posted the same day as : Why Don't They Make BIGGER CPUs? - (Techquickie) https://www.youtube.com/watch?v=8JAWz9Da5og https://www.youtube.com/watch?v=8JAWz9Da5og Now I know its just a LinusTechTips level video, but I guess they havent heard of "Wafer Scale Engine", and I hadn't either, but now this proves the video is already obsolete…
- segfaultbuserr 6y agoNot obsolete yet, the technology is not mature enough, and the previous arguments against WSI are still valid. People have been trying Wafer-Scale Integration [0] since the 1970s, there was quite some hype of building a "super chip" back at that time [1], but all efforts failed miserably. Cerebras' success is only the beginning, even if this approach is workable (which remains a question), there's still at least a decade to go from a HPC-specific chip to a general-purpose chip. Another possibility is that WSI will forever be a technology used in massively-parallel computers. [0] https://en.wikipedia.org/wiki/Wafer-scale_integration https://en.wikipedia.org/wiki/Wafer-scale_integration [1] Giant microcircuits for superfast computers, Popular Science, 1984. https://books.google.com/books?id=eAAAAAAAMBAJ&pg=PA66 https://books.google.com/books?id=eAAAAAAAMBAJ&pg=PA66
- gridlockd 6y agoAnother possibility is that the company will just fail, because it can not offer an advantage over commodity chips.
- segfaultbuserr 6y agoI already mentioned it, > even if this approach is workable (which remains a question) No need to repeat.
- jecel 6y agoI think it is important to separate the several technical failures in wafer scale from the commercial one: Sinclair's Anamartic worked exactly as planned but the prices for conventional hard disks began to drop and that caused investors to pull out. http://www.computinghistory.org.uk/det/3043/Anamartic-Wafer-Scale-160MB-Solid-State-Disk/ http://www.computinghistory.org.uk/det/3043/Anamartic-Wafer-... The problem is that the real project was a massively parallel computer just like the Cerebras (scaled to 1989 technological limits) and the disk replacement was just a way to develop the needed techniques and finance further developments. If the investors had had a little more patience then computing in the 1990s might have been a bit more interesting.
- 11thEarlOfMar 6y agoIt must have some redundancy in order to survive process errors. Does it have to be defect-free to yield?
- segfaultbuserr 6y agoAnswered in the PDF [0] that HN user "aveni" linked below, https://www.hotchips.org/hc31/HC31_1.13_Cerebras.SeanLie.v02.pdf https://www.hotchips.org/hc31/HC31_1.13_Cerebras.SeanLie.v02... > Redundancy is Your Friend > Uniform small core architecture enables redundancy to address yield at very low cost > Design includes redundant cores and redundant fabric links > Redundant cores replace defective cores > Extra links reconnect fabric to restore logical 2D mesh
- deleted 6y ago[deleted]
- etaioinshrdlu 6y agoThe most interesting part of this to me is the staggering amount of SRAM on the processor. I wouldnt have thought this amount of sram to be practical from a size and cost standpoint compared to DRAM.
- Tuna-Fish 6y agoYou cannot make DRAM on the same process with competitive, high-end logic, and they want to embed a lot of buffers on chip to reduce communication power use, so SRAM is the only game in town.
- skavi 6y agoIntel had eDRAM (Crystal Well) for a while as the cache for their integrated Iris graphics.
- etaioinshrdlu 6y agoFrom wikichip: The eDRAM silicon die is separate from the main Haswell die but is packaged together with it.
- skavi 6y agoHuh, I should have done more reading.
- Tuna-Fish 6y agoCrystal Well was only eDRAM in the marketing sense. There have been other, actual eDRAM technologies that can be manufactured on some logic processes, however none of them are compatible with the state of the art TSMC 7nm logic process.
- modeless 6y agoYeah, but it's not enough. The whole chip doesn't have enough SRAM to hold a tenth of GPT-3, let alone GPT-4. These are the kinds of models you would hope a chip like this would help with, but as soon as the model gets too big to fit in the SRAM then the advantage goes way down. And I think the models of the future will continue to get much, much bigger.
- debbiedowner 6y agoSo does this beat 8 V100s in a DGX? And if yes what TPU v3-n is it comparable to?
- trhway 6y agoby transistor count, by energy consumption and i'd expect by the price it is an equivalent of 3 x 16 V100 DGX/Lambdasystems. Can it beat such a minicluster? Vertical vs. horizontal scaling seems to be still an open question for DL. It looks like the horizontal - ie. V100 approach in this case - would naturally have advantage over the vertical when it comes to sheer scale, i.e. like the maximum possible size of model.
- Rafuino 6y agoDid anyone put their first gen to use? Saw that huge chip last year (or maybe the year before at this point... time doesn't exist anymore...), but I never heard of actual users or buyers.
- aveni 6y agoBased on press, there are already some systems being deployed (Argonne National Labs, Pittsburgh Supercomputing Center).
- Cthulhu_ 6y agoI'm sure companies that do a lot with AI - the FAANGs, universities - would have their hands on these to see if they're viable for their businesses, either for their own use or to rent out to customers through their cloud offerings. But they don't really advertise with it, probably corporate secrecy. I think it has the same customer base as the quantum processors of today.
- petra 6y agoIs there some recent cloud offering that might be using cerebras systems ?
- kregasaurusrex 6y agoWatched the talk at Hot Chips today[0]- 400,000(!) cores on a single wafer is absolutely remarkable! There was a slide not posted in which there were virtual segments of the wafer designated for each step, and the example given by the presenter showed a perfectly in-order workflow execution. [0] https://www.anandtech.com/show/14758/hot-chips-31-live-blogs-cerebras-wafer-scale-deep-learning https://www.anandtech.com/show/14758/hot-chips-31-live-blogs...
- lsb 6y agoTheir 2020 announcement is 850K cores, which is even more impressive!
- mvn9 6y agoHow much does it cost or rather how much does that entire rack with all the supporting systems cost?
- epicureanideal 6y agoWhat does this mean for Moore's law being dead or not? Anything?
- throwaway189262 6y agoNah, were near the scaling limit of silicon. And huge chips are super expensive and impractical. One day we'll find an easier way to print chips. Won't end moore's law but may shift algorithm implementation back to hardware after decades of software eating everything. Medium term, they'll get a new lease on moore's law by switching off silicon to smaller atoms. And/or switching to something that handles heat better so we can go 3D. This is already happening in power electronics with GaN. My bet is on this tech slowly spreading into digital logic
- spicyramen 6y agoInterested in how Google TPU new generation compares to this one. Is this comparison even reasonable?
- blueblisters 6y agoMight it not be more optimal to attach the individual GPU dielets onto a silicon interposer to avoid the yield issues that are inherent when making a large monolithic chip, and save some of the overheads of dealing with dead dielets? I think I read about some researchers/companies going that route but haven't kept up.
- lsb 6y agoPart of the genius of the Cerebras on-chip routing is that it can route around malfunctioning dielets.
- gridlockd 6y agoEven then, wouldn't it be better to just use commodity chips on an interposer? I don't actually know, but that seems to be the route the industry is going and economics of scale will likely make it the winning approach.