8 ms·
Intel announces the Aurora supercomputer has broken the exascale barrier
- Gerlo 2y agoAurora was supposed to go into production years ago and be the first exascale supercomputer. Intel is dog shit, so this kept getting delayed. Now they're going for the two exaflop mark. It's pathetic that Aurora is only now benchmarking over an exaflop, and it's even more pathetic than this is apparently newsworthy.
- JonChesterfield 2y agoThe link is Intel's own website. Has excellent quotes on it like: > Why It Matters: Designed as an AI-centric system from its inception Announcement was in 2015. I'm curious whether Argonne are pleased with it.
- Gerlo 2y ago>Announcement was in 2015. I'm curious whether Argonne are pleased with it. All I'll say is that you can probably guess how they feel about it given the context.
- alephnerd 2y agoIntel was chosen because the DoE wanted to foster an entire ecosystem of GPU vendors, because having a single vendor is a MASSIVE bottleneck. Intel, AMD, and Nvidia were all vendors on Exascale projects [0] > Intel is dog shit Intel has issues with execution, but their engineers are still top notch. They are the last American company to actually do semiconductor fabrication, and only fell behind TSMC and Samsung in fabrication only 6-7 years ago because they didn't choose to invest in EUV lithography instead of other methods. [0] - http://helper.ipam.ucla.edu/publications/nmetut/nmetut_19423.pdf http://helper.ipam.ucla.edu/publications/nmetut/nmetut_19423...
- CaliforniaKarl 2y agoCase in point, Intel’s discrete consumer GPUs: They’re making major gains with each driver release, and I hope to see them seriously competing with nvidia in the gaming market.
- hi-v-rocknroll 2y agoA solution can always be achieved by spending more money.™ Rather than engineering for economic scale, engineer for maximizing billable hours and make yourself indispensable. - Consultant's credo With all of the taxpayer money they've wasted so far, they could've bought zillions of Cerebras WSE-3 and exceeded 10 exaflops.
- CaliforniaKarl 2y agoMore context: This is related to today's release of the Spring Top 500 list (https://news.ycombinator.com/item?id=40346788 https://news.ycombinator.com/item?id=40346788). Aurora rated 1,012.00 PetaFLOPS/second Rmax, and is in 2nd place, behind Frontier. In the November 2023 list, Aurora was also in second place, with an Rmax of 585.34 PetaFLOPS/second. See https://www.top500.org/system/180183/ https://www.top500.org/system/180183/ for the specs on Aurora, and https://www.top500.org/system/180047/ https://www.top500.org/system/180047/ for the specs on Frontier. See https://www.top500.org/project/top500_description/ https://www.top500.org/project/top500_description/ and https://www.top500.org/project/linpack/ https://www.top500.org/project/linpack/ for a description of Rmax and the LINPACK benchmark, by which supercomputers are generally ranked. The Top 500 list only includes supercomputers that are able to run the LINPACK benchmark, and where the owner is willing to publish the results. The jump in Aurora's Rmax scope is explained by Aurora's difficult birth. https://morethanmoore.substack.com/p/5-years-late-only-2 https://morethanmoore.substack.com/p/5-years-late-only-2 (published when the November 2023 list came out) has a good explanation of what's been going on.
- klysm 2y agoWhat does FLOPS/second mean? Isn’t FLOPS already per second? Are they accelerating?
- dragonwriter 2y agoYeah, the top500 pages cited use Flop/s (apparently using Flop for “Floating point operations” – not sure which “o” and “p” are used), I’ve could swear I’ve seen FLOPS and seen it expanded specifically as “FLoating point Operations Per Second” when I first encountered it, FLOPS/s seems to be using “FLOPS” like the “Flop” above (probably as “FLoating point OPerationS”, in which case the “/s” makes sense.)
- kortilla 2y agoSome people treat FLOPS as “FLoating point OPerationS”.
- deleted 2y ago
- JonChesterfield 2y agoOn target to be somewhat slower than Frontier at double the power consumption.
- modeless 2y agoOff topic but "Break the ___ barrier" has got to be my least favorite expression that PR people love. It's a "tell" that an article was written as meaningless pop science fluff instead of anything serious. The sound barrier is a real physical phenomenon. This is not. There's no barrier! Nothing was broken!
- alephnerd 2y agoThe Exascale barrier is an actual barrier in HPC/Distributed Systems. It took 15-20 years to reach this point [0] A lot of innovations in the GPU and distributed ML space were subsidized by this research. Concurrent and Parallel Computing are VERY hard problems. [0] - http://helper.ipam.ucla.edu/publications/nmetut/nmetut_19423.pdf http://helper.ipam.ucla.edu/publications/nmetut/nmetut_19423...
- dghlsakjg 2y agoThe comment isn't saying that the benchmark isn't useful. They are saying that there is no 'barrier' to be broken. The sound barrier was relevant because there was a significant physical effects to overcome specifically when going trans-sonic. It wasn't a question of just adding more powerful engines to existing aircraft. They didn't choose the sound barrier because its a nice number, it was a big deal because all sorts of things behaved outside of their understanding of aerodynamics at that point. People died in the pursuit of understanding the sound barrier. The 'exascale barrier', afaict, is just another number chosen specifically because it is a round(ish) number. It didn't turn computer scientists into smoking holes in the desert when it went wrong. This is an incremental improvement in an incredible field, but not a world changing watershed moment.
- trueismywork 2y agoIt is definitely not incremental. If you watch some talk on Gordon Bell prize for Frontier, you see that the dynamics have changed completely. Data, software stack, I/O have suddenly become bottlenecks in multiple places. So yes, it is a watershed moment..
- Harmohit 2y agoSerious question: my understanding of HPC is that there are many workloads running on a given supercomputer at any time. There is no singular workload that takes up the entire or most of the resources of a supercomputer. Is my understanding correct? If yes, then why is it important to build supercomputers with more and more compute? Wouldn't it be better to build smaller systems that focus more on power/cost/space efficiency?
- sseagull 2y agoYou are generally correct, however there are workloads that do use larger portions of a supercomputer that wouldn't be feasible on smaller systems. Also, I guess I'm not sure what you mean by "smaller systems that focus more on power/cost/space". A proper queueing system generally efficiently allocates the resources of a large supercomputer to smaller tasks, while also making larger tasks possible in the first place. And I imagine there's somewhat an efficiency of scale in a large installation like this. There are, of course, many many smaller supercomputers, such as at most medium to large universities. But even those often have 10-50k cores or so. (In general, efficiency is a consideration when building/running, but not of using. Scientists want the most computational power they can get, power usage be damned :) ) edit: A related topic is capacity vs. capability: https://en.wikipedia.org/wiki/Supercomputer#Capability_versus_capacity https://en.wikipedia.org/wiki/Supercomputer#Capability_versu...
- tkuraku 2y agoWhile in general there can be many smaller workloads running in parallel. However, periodically the whole supercomputer can be reserved for a "hero" run.
- deleted 2y ago[deleted]
- bitfilped 2y agoMost of the time yes, HPC systems are shared among many users. Sometimes though the whole system (or near it) will be used in a single run. These are sometimes referred to as "hero runs" and while they're more common for benchmarking and burn-ins there are some tightly-coupled workloads that perform well in that style of execution. It really depends on a number of factors like the workloads being run, the number of users, and what the primary business purpose of the HPC resource is. Sites that have to run both types of jobs will typically allow any user to schedule jobs most of the time but then pre-reserve blocks of time for hero runs to take place where other user jobs are held until the primary scheduled run is over.
- mperham 2y agoEvery single paragraph contains the word "AI".
- porphyra 2y agoI feel like a lot of the big GPU clusters that companies like Meta are using would score highly on the benchmark, even though Linpack seems to be based on fp64 which the H200 is kinda sucky at. However, there's only one GH200-based machine (Alps, with 2688 GH200 nodes) in the top 10 even though I'm fairly sure it's quite a bit smaller than whatever Meta is training Llama 3 on. Is there any reason why we don't see those show up in TOP500?
- pixl97 2y agoThey are busy running training/inference on them and haven't benchmarked them?
- porphyra 2y agoSo these benchmark lists like TOP500 are just fundamentally flawed because the serious players are too busy doing actual work than to participate?
- pixl97 2y agoI mean there is no rule if you buy a supercomputer that you have to benchmark it and submit the results. This said, in the days before AI the number and type of players that had this amount of compute were also commonly the types to submit their scores to said benchmark lists.
- CaliforniaKarl 2y agoMost players that had this amount of compute also tend to pay less than the equivalent corporate wage, and so besides being a good way to stress-test your cluster (prior to general availability), it gives you the positive feeling of “I helped make this, I help run this.”
- kkielhofner 2y agoI work with a couple of the national labs and the talent there is generally incredible. "I went to school for > 25 years so my life's work could be selling more ads" isn't a motivator for most of these people.
- gerdesj 2y ago"and is the fastest AI system in the world dedicated to AI for open science" Cool. Please ask it to sue for peace in several parts of the world, in an open way. Whilst it is at it, get it to work out how to be realistically carbon neutral. I'm all in favour of willy waving when you have something to wave but in the end this beast will not favour humanity as a whole. It will suck up useful resources and spit out some sort of profit somewhere for someone else to enjoy.
- cmdrk 2y agoThey’re simply latching onto the AI buzzwords for the good press. Leadership class HPCs have been designed around GPUs for over a decade now, it just so happens they can use those GPUs to run the AI models in addition to the QCD or Black hole simulations etc that they’ve been doing for ages.
- alephnerd 2y agoNvidia's entire DGX and Maxwell product line was subsidized by Aurora's precursor, and Nvidia worked very closely with Argonne to solve a number of problems in GPU concurrency. A lot of the foundational models used today were trained on Aurora and its predecessors, as well as tangential research such as containerizarion (eg. In the early 2010s, a joint research project between ANL's Computing team, one of the world's largest Pharma companies, and Nvidia became one of the largest customers of Docker and sponsored a lot of it's development)
- kkielhofner 2y agoIt's much more than that. For example, I have an "AI" project on Frontier. The process was remarkably simple and easy - a couple of Google Meets, a two page screed on what we're doing, a couple of forms, key fob showed up. Entire process took about a month and a good chunk of that was them waiting on me. Probably half a days work total for 20k node hours (four MI250x per node) on Frontier for free, which is an incredible amount of compute my early resource constrained startup would have never been able to even fathom on some cloud, etc. It was like pulling teeth to even get a single H100 x8 on GCP for what would cost at least $250k for what we're doing. And that's with "knowing people" there... These press releases talking about AI are intended to encourage these kinds of applications and partnerships. It's remarkable to me how many orgs, startups, etc don't realize these systems exist (or even consider them) and go out and spend money or burn "credits" that could be applied to more suitable things that make more sense. They're saying "Hey, just so you know these things can do AI too. Come talk to us." As an added bonus you get to say you're working with a national lab on the #1 TOP500 supercomputer in the world. That has remarkable marketing, PR, and clout value well beyond "yeah we spent X$ on $BIGCLOUD just like everyone else".
- gary_0 2y agoAccording to Wikipedia[0] it uses 38.7MW of power, beating Fugaku (29.9MW) to be #1 in the TOP500 for power consumption. [0] https://en.wikipedia.org/wiki/Aurora_(supercomputer) https://en.wikipedia.org/wiki/Aurora_(supercomputer)
- ingen0s 2y agoModern day Manhattan Project
- greggsy 2y agoThat line seems to emerge every time a new supercomputer is commissioned on the US… in fact, the naming of Los Alamos’ new Crossroads, along with its predecessor, Trinity, have obvious roots in IS nuclear testing history. https://discover.lanl.gov/news/0830-crossroads/ https://discover.lanl.gov/news/0830-crossroads/
- Mikhail_K 2y agoImagine a Beowulf cluster of those!
- jp57 2y agoWas there actually a barrier at exascale? I mean, was this like the sound barrier in flight where there is some discontinuity that requires some qualitatively different approaches? Or is the headline just a fancy way of saying, "look how big/fast it is!"
- colechristensen 2y agoIt's a milestone, not a barrier.
- chungy 2y agoBarrier sounds cooler in a press release than "we made it fast enough to surpass an arbitrary benchmark"
- saltcured 2y agoYeah. Back in the 90s, "terascale" was the same kind of milestone buzzword that was being thrown around all the time. Because of that, I felt a bit of nostalgia when I first saw consumer-accessible GPUs hitting the 1 TFLOP performance level, which now I suppose qualifies as a cheap iGPU.
- colechristensen 2y agoI have no problem with “exascale” or whatever, it’s just not a barrier, there was nothing in particular to overcome to get there, it’s just a signpost along the normal path forwards. Unlike the sound barrier in flight, it was for a while thought impossible to fly faster than sound and there are indeed models of subsonic flight that have an infinity in them at the speed of sound. It indeed took new models and radically different designs to fly faster than sound. Making this computer do a certain number of calculations didn’t involve any such problem to overcome.
- magicalhippo 2y agoOne thing is having a bunch of computers. Another thing is to get the working efficiently on the same problem. While I'm not sure exascale was something like the sound barrier, I do know a lot of hard work has been done to be able to efficiently utilize such large clusters. Especially the interconnects and network topology can make a huge difference in efficiency[1] and Cray's Slingshot interconnect[2], used in Aurora[3], is an important part of that[4]. [1]: https://www.hpcwire.com/2019/07/15/super-connecting-the-supercomputers-innovations-through-network-topologies/ https://www.hpcwire.com/2019/07/15/super-connecting-the-supe... [2]: https://www.nextplatform.com/2022/01/31/crays-slingshot-interconnect-is-at-the-heart-of-hpes-hpc-and-ai-ambitions/ https://www.nextplatform.com/2022/01/31/crays-slingshot-inte... [3]: https://www.alcf.anl.gov/aurora https://www.alcf.anl.gov/aurora [4]: https://arxiv.org/abs/2008.08886 https://arxiv.org/abs/2008.08886
- croes 2y agoWasn't it supposed to be a 2 exascale system?