13 ms·
Cerebras’ new monster AI chip adds 1.4T transistors
- TedDoesntTalk 6y ago> the size ... [is] ... 46,225 square millimeters According to google calculations, this is just under 6 square feet.
- Phil-bitplex 6y agoI think it might be closer to 215mm x 215mm or 0.49 square feet. Though that is unimaginably large to me for a silicon chip. I can't comprehend it - the engineering behind it would be incredible.
- deathanatos 6y agoNo…? https://www.google.com/search?q=46%2C225+square+millimeters+in+square+feet https://www.google.com/search?q=46%2C225+square+millimeters+... > 0.49756176 square foot
- fredfoobar 6y agoTime for an AI winter I guess.
- Logon90 6y agoJust from the headline you can see the irrelevance of this chip. Who talks about transistor count as a proxy of performance?
- streetcat1 6y agoYou do realize that AI crossed human expert performance in NLP / Vision tasks ?
- semi-extrinsic 6y agoExactly how does one outperform a human expert in natural language processing?
- pulse7 6y agoParent probably meant that it outperformed a human expert in some specific task in the area of natural language processing - for example the task of converting a spoken language into a written language...
- zetazzed 6y agoSee, if you were an AI, you would understand EXACTLY what the poster means by this.
- twic 6y agoMaybe the gibberish GPT-3 spits out is actually true, and our puny monke brains are just too weak to understand it.
- PeterisP 6y agoThey don't really outperform human experts on real tasks yet (no matter what some superGLUE or other benchmark shows); but in general, once a system can solve a particualr task well, it would be plausible to outperform human experts simply by not making random errors. If we have multiple human experts annotate a NLP task and measure inter-annotator agreement, it will be far from 100%; part of that will be genuine disagreements or fuzzy gray area, but part of the identified differences will be simply obviously wrong answers given by the experts - everyone makes mistakes. The same applies for many other domains - business process automation, data entry, etc; no employee will produce error-free output in a manual process, no matter how simple and unambiguous the task is. And for simpler tasks the computer can easily make less mistakes than a human - especially if you measure the human reliability not for a few minutes of focus, but for a whole tedious working day.
- firebaze 6y agoI'm not sure if you're assuming best intentions. If you do, it'd be nice to provide sources - to my knowledge, vision under non-optimal conditions (rain, snow, sunlight ahead) is only partially solved by resorting to sensors resistant to the disturbance. I'd be glad to learn I'm wrong.
- fredfoobar 6y agoTBH, I believe we need another AI winter to come up with something better than the AI "black boxes" we have right now.
- king_magic 6y agoNo, it hasn't. Not even close. AI has "crossed human expert performance" on extremely narrow NLP/CV tasks. AI is still light years away from human-level performance.
- anthk 6y agoGPT-3 is not even close to a minimal and coherent text adventure made with Inform6 from a novice, even if it's written by a non-native English speaker. Those networks didn't match "Detective", a crappy story written by a 12yo.
- trhway 6y ago>Time for an AI winter with AI chips burning 15KW? No chance for the winter in sight. Some chances for AI hell though.
- fredfoobar 6y agoThis statement was made for experimental purposes, to understand HN biases.
- systemvoltage 6y agoMost interesting aspect of wafer-scale manufacturing is yield. Even if we have 95% chip yield, as the chip size approaches the wafer-level dimensions, I don't know off top of my head what the math would be but it is going to plummet drastically. My guess is that they're handling this in the chip logic. Building resiliency by turning off cells in the wafer that didn't yield. That begs the question, how are they probing them? A probe card of the size of the wafer is unheard of. How are they running validation? Pretty mindblowing to say the least!
- belval 6y agoNot an expert in chip manufacturing but my guess is that they just disable the parts that don't work and their big numbers represent ~80% of the actual number of transistors in the wafer because they account for that manufacturing loss.
- KETpXDDzR 6y agoCorrect. You already have the same with modern CPUs and GPUs. You just disable the defect parts. Obviously, that sounds way easier than it is.
- vkazanov 6y agoDisabling parts of the chip? The secret sauce then is to make it defect-proof
- gsnedders 6y agoYup. This. From https://www.anandtech.com/show/16626/cerebras-unveils-wafer-scale-engine-two-wse2-26-trillion-transistors-100-yield https://www.anandtech.com/show/16626/cerebras-unveils-wafer-...: > Cerebras achieves 100% yield by designing a system in which any manufacturing defect can be bypassed – initially Cerebras had 1.5% extra cores to allow for defects, but we’ve since been told this was way too much as TSMC's process is so mature.
- baybal2 6y agoYep, not all semiconductor defects can be repaired, and at such scale repair circuits themselves will need to be redundant.
- LASR 6y agoThis is an interesting approach. Are there any benchmark/performance indicators for these wafer-scale chips?
- tandr 6y agoHow do they do heat management and dissipation on such a big wafer? I can imagine some different parts heats up differently, putting a mechanical strain on wafer, and leading to cracks.
- punnerud 6y agoA large portion of the article is answering most of this
- tandr 6y agoNo, it does not. There is a paragraph _talking_ about it, but not much info still.
- NortySpock 6y ago"Carefully." The article mentions a year of engineering went into dealing with the entire wafer thermally expanding under load.
- addaon 6y agoIf you look at the die shots (wafer shots?), you'll notice small holes a few millimeters across spaced roughly at reticle spacing. Those are drilled holes to allow through-wafer liquid cooling. With liquid cooling not just around but through the wafer, the temperature differential is minimized.
- tandr 6y agothank you
- etaioinshrdlu 6y agoHow much memory is on the chip, and what kind is it? Under what circumstances does the chip need to access external memory? What type of communication interfaces does this chip have? Also, if the chip is the size of a wafer, is it appropriate to call it a Chip?
- verdverm 6y agoTom's Hardware has some nice tables comparing the specs: https://www.tomshardware.com/news/cerebras-wafer-scale-engine-2-worlds-largest-chip-7nm-850000-cores https://www.tomshardware.com/news/cerebras-wafer-scale-engin... (more than the IEEE) This thing pulls 15-20kW of juice!
- teruakohatu 6y agoI did a double that when I realised that was Kilowatts not Watts. This chip uses more energy in an hour than the average household (in my country at least) does in a day. It may be a very large wafer but dissipating that heat is still very impressive.
- kllrnohj 6y agoIt sounds like a lot but it almost isn't? Like this is ~50x bigger than an Nvidia A100, and the A100 pulls up to 400w. 50 * 400 ~= 20kW. So in terms of thermal density it's in-line with existing GPUs. That said, I'd be fascinated to see the cooling solution. Is it just a massive copper heatsink & a boatload of airflow? Typical approaches of using heatpipes to expand the heatsink won't really work with something this big after all. Or is it a massive waterblock with multiple inlets/outlets so it can hit up a stack of radiators? How do they get even mounting pressure across that large of an area?
- mvanaltvorst 6y agoThere's an image on their website[1], pretty huge water pumps. "To solve the 70-year-old problem of wafer-scale, we needed not only to yield a big chip, but to invent new mechanisms for powering, packaging, and cooling it. The traditional method of powering a chip from its edges creates too much dissipation at a large chip’s center. To prevent this, CS-2’s innovative design delivers power perpendicularly to each core. To uniformly cool the entire wafer, pumps inside CS-2 move water across the back of the WSE-2, then into a heat exchanger where the internal water is cooled by either cold datacenter water or air." [1]: https://cerebras.net/product/ https://cerebras.net/product/
- bhewes 6y agoThis thing reminds me of Daniel Hillis's "The Connection Machine". Just 35 years later.
- xhrpost 6y agoHow can the chip itself consume that kind of power? Or is the 15kw value for the entire unit? That's like 10 residential space heaters all turned to max. I'm surprised that much heat could be dissipated over such a small surface area. Does it use refrigerant for cooling? If my math is correct, if you had a 6500BTU window air conditioner, you'd need 8 of them to move the heat from this chip.
- philjohn 6y agoWell, a single 7nm CPU (AMD Ryzen) can pull down 95 odd watts at peak, granted, the IO die is a fair bit of that as it's on an older process, but if you extrapolate that out across a giant wafer it's "only" 157 Ryzen 7's.
- YetAnotherNick 6y agoIt would melt if you just do air cooling. I did some math for liquid nitrogen which has heat of vaporization as 200 kJ / kg. It would boil 3600*15/200 = 270 kg of nitrogen every hour. Just insane.
- virtuallynathan 6y agoIt’s water cooled for sure, perhaps with chilled water, but all integrated into the 15U chassis. There are cutaway images of the v1 system online.
- KETpXDDzR 6y agoI think I once saw one of the founders with a wafer in an In n' out with a potential investor. Looking at what Apple achieved with their M1A and the demand for "AI" - or training neutral networks, what it really is - they have a lot of potential. At least as long as the AI bubble doesn't burst.
- arisAlexis 6y agoHow can the future burst? It's like saying medicine will burst or physics
- lainga 6y agoI don't know, expert systems haven't been so hot lately...
- arisAlexis 6y agoIt's very surprising what people with downvote power do but it's *** to downvote anyone that disagrees. Anyway expert systems were a precursor to the technology and they got replaced much like new physics replace old physics.
- ASalazarMX 6y agoDisagreeing doesn't mean you're right. AI is not equivalent to the future, and it will burst if it stalls and another AI winter cools the current hype cycle.
- jefft255 6y agoIt's because you made a false analogy. AI isn't literally "the future". Billions of $ are being invested in deep-learning focused AI right now (which you call "the future"), and yes it could be a bubble and it could burst. You can disagree, but it's still a sensible thing to predict.
- arisAlexis 6y ago
- zitterbewegung 6y agoMuch better article from anandtech at https://www.anandtech.com/show/16626/cerebras-unveils-wafer-scale-engine-two-wse2-26-trillion-transistors-100-yield https://www.anandtech.com/show/16626/cerebras-unveils-wafer-...
- wyxuan 6y agoHere's a video made by the author, Ian Cuttress, which goes into more detail as well. https://www.youtube.com/watch?v=FNd94_XaVlY https://www.youtube.com/watch?v=FNd94_XaVlY
- smithza 6y agoThis is very cool. The explanation of how yield/defects was interesting: they can bypass cores with defects due to channeling and account for the statistical defects, allowing them to have a 100% yield.
- m463 6y agoThis is how pretty much all large chips are designed nowadays. The perfect chips are expensive higher-capacity / faster chips, the less perfect chips are cheaper lower capacity and/or slower.
- XCSme 6y agoI laughed at the "arm+leg" official cost of the CPU. EDIT: Or GPU, or whatever it is.
- cpr 6y agoInteresting--the placement of code-to-processor so that things will be done roughly at the same time sounds a lot like the VLIW compiler problem of scheduling execution units so things are available at exactly the right time, without hardware interlocks.
- durst 6y agoDoes anyone here have firsthand experience using the compiler? Can you give a rough approximation of performance tuning with the compiler compared to performance tuning with compilers targeting the tensor cores on an A100 or a TPU?
- terafo 6y agoIs there any information on how this thing performs on actual AI workloads? It is priced similarly to a dozen of DGX A100s, but is it faster on something like training big transformer models(such as CLIP or GPT-3)?
- 01100011 6y agoThis thing is awesome and, as someone working for a competitor, kind of scary. I applaud their approach though. I think we're a couple years off from it, but we'll probably see wider adoption of larger silicon, with more specialized functional units, which are used with a lower duty cycle to manage heat. If nothing else, they're probably developing some good IP and techniques to handle other sorts of ultra-mega-insane-scale-integration. I wonder what their software stack looks like. Can they support the sort of virtualization and sharing you'd want to keep this expensive beast fully utilized 24/7?
- ohazi 6y agoLSI, VLSI, UMISI I like it! In previous articles they've gone into some detail about how they deal with reticle limits, jumping over the scribe line area, and other n stuff. Between that, chiplets, HBM-style die stacks, etc... the developments here have been more interesting than I expected.
- zetazzed 6y agoI wonder what the dev story looks like for these? I know they say "just use TF/Pytorch" but surely developers need to actually test stuff out on silicon and run CI on code... do they offer a baby version for testing?
- AlexCoventry 6y agoImagine a beowulf cluster of these.
- faichai 6y agoThis made me chuckle!
- michelpp 6y agoThis thing needs a GraphBLAS[1] implementation yesterday. 100 billion edge graphs and up are the new norm. This monster could smoke the competition if the implementation was tuned right! [1] http://graphblas.org http://graphblas.org
- blueyes 6y agoCreating the ecosystem of both software and adjacent hardware for wafers this size is the real challenge for a company like Cerebras (which is doing amazing work). At first, they thought they just needed to make a chip 56x the size of its predecessor, and somehow get around the issue of defects and yield. After they solved those problems (which blocked Gene Amdahl, among others), they found they needed to bring an entire ecosystem into being to work with their hardware.
- michelpp 6y agoAgreed, that's why I think the GraphBLAS would be such a great fit for this hardware. The ecosystem is growing pretty fast. There are, for example Python bindings, you can do sparse 'A @ B' on millions of elements in parallel with this wafer-chip. MATLAB 2021a now has GraphBLAS built in, you could drive this thing directly from your notebooks. I'm sure there's a compiler and low level primitives to really get the maximum performance out of it, but the trade-off maybe worth it in many cases to do it using an abstraction like the linear algebra approach.
- lastrajput 6y agoWho said Moore’s law is dead?
- miohtama 6y agoIt’s not dead, but melting.
- MangoCoffee 6y agoits not a law but an observation.
- sillysaurusx 6y agoI’m bearish on new hardware for AI training. The most important thing is the software stack, and thus far everyone has failed to support pytorch in a drop-in way. The philosophy here seems to be “if we build it, they’ll buy it.” But suppose you wanted to train a gpt model with this specialized hardware. That means you’re looking at two months of R&D minimum to get everything rewritten, running, tested, trained, and with an inferencing pipeline to generate samples. And that’s just for gpt — you lose all the other libraries people have written. This matters more in GAN training, since for example you can find someone else’s FID implementation and drop it in without too much hassle. But with this specialized chip, you’d have to write it from scratch. We had a similar situation in gamedev circa 2003-2009. Practically every year there was a new GPU, which boasted similar architectural improvements. But, for all its flaws, GL made these improvements “drop-in” —- just opt in to the new extension, and keep writing your gl code as you have been. Ditto for direct3d, except they took the attitude of “limit to a specific API, not arbitrary extensions.” (Pixel shader 2.0 was an awesome upgrade from 1.1.) AI has no such standards, and it hurts. The M1 GPU in my new Air is supposedly ready to do AI training. Imagine my surprise when I loaded up tensorflow and saw that it doesn’t support any GPU devices whatsoever. They seem to transparently rewrite the cpu ops to run on the gpu automatically, which isn’t the expected behavior. So I dig into Apple’s actual api for doing training, and holy cow, that looks miserable to write in swift. I like how much control it gives you over allocation patterns, but I can’t imagine trying to do serious work in it on a daily basis. What we need is a unified API that can easily support multiple backends — something like “pytorch, but just enough pytorch to trick everybody” since supporting the full api seems to be beyond hardware vendors’ capabilities at the moment. (Lookin’ at you, google. Love ya though.)
- whimsicalism 6y agoI'm on board with you that there should be a "drop-in" cross support of these chips, but pytorch is at a way higher abstraction level than what should be commonly supported.
- socialdemocrat 6y agoMaybe people will get to their senses and switch to Julia instead of having to waste all these time on Python bindings.
- noobydoobydoo 6y agoWhat does the programming model look like for one of these (like at the assembly level)? I’m not even sure what to google.
- durst 6y agoThe article states that the programming model is TensorFlow or PyTorch. However, I'm not clear on the details of customizing TensorFlow or PyTorch code to run efficiently on this chip.
- userbinator 6y agoI suspect it's more like an FPGA - massively parallel structures that don't fit traditional "programming" well. Incidentally, FPGAs are also insanely expensive at the high end.
- syntaxing 6y agoSuper curious, how much do one of these cost? Like 10K,100K, or 1M range?
- thunkshift1 6y agoFirst gen was ~ 2.5M usd per machine as per the video link
- LegitShady 6y agoAny article or headline talking about transistor count is just a puff piece. The performance matters, the number of transistors never does, and I have never purchases anything based on the number of transistors.
- mousepilot 6y agoI'm not going to buy one, but its more because I don't have several million dollars to spare.
- LegitShady 6y agoI get that, but it's just a thing I've noticed about tech puff pieces. If the transistor count is front and center it's probably not worth reading.
- v8dev123 6y agoWhy it's called as "AI"?
- macksd 6y agoDesigned for deep learning
- intrasight 6y agoInteresting consider that if this "chip" was manufactured with the feature size of the MC68000, the chip would be about 1/4 mile square.
- intrasight 6y agoOr another way to grok the process shrinkage is to consider that the MC68000, if manufactured with current process dimensions, would fit within the space of a single gate of the original process.
- jzer0cool 6y agoThis is the first I'm coming across Cerebras and interested here to understand more. Are these already in use integrated in some pc/servers or is it some dedicated server which you need to integrate into some motherboard, ...
- kumarski 6y agoPolyhedral LLVM compiler optimization > Quantum Computing Technology Well at least on relevance* to active problems.
- lxe 6y ago> That chip needed some 20,000 amps of current fed to it from one million copper connections to a fiberglass circuit board atop the wafer. Is the 20,000 amp number not an error?
- Karliss 6y ago20000 amps seems plausible. A regular desktop CPU operates at 1-2 Volts and power consumption of 100-200 watts which means 50-200 Amps. Comparison table shows nvidia A100 which has max power consumption of 400W. This has roughly 50 times the transistors and 50 surface area so having 50 times the power consumption 50*400=20000W and same order of magnitude of current seems possible.
- magicalhippo 6y agoThey also say the peak power draw is 23kW. Now 23kW/20kA = 1.1V which is about what my NVIDIA 2080 GPU uses at load. So that does indeed seem to work out. Of course 20kA is pretty insane, would be interesting to see just how they feed the power to this thing. Bond wires going to the middle of the chip? And the local regulator must be a beast, or rather lots of beasts.
- ksec 6y agoFor those interested, here is an article [1] on glimpse of how it is cooled. Although I still dont understand how this is possible though. 20kW. [1] https://www.eetimes.com/powering-and-cooling-a-wafer-scale-die/# https://www.eetimes.com/powering-and-cooling-a-wafer-scale-d...
- Bancakes 6y agoMy canvas fingerprint addon went berserk soon as I visited that link. 45 scripts and 14 fonts as well. Thanks for the link, though.