10 ms·
EU Grabs ARM for First ExaFLOP Supercomputer
- zzbn00 3y agoCan report Jülich is not near Munich.... Anyway will be fascinating to see how the SiPearl chip works out.
- solarist 3y agoThey mixed up Garching with Jülich.
- eunos 3y agoYea, much closer to Aachen in NRW. Side note, it's a small town without campus, I wonder why dont they locate the center in Aachen or Koln.
- zzbn00 3y agoMy vague memory is that there was/is a military or government site nearby which is why the research centre is also there.
- b3orn 3y agoBut there's a big research center next to the small town. https://en.wikipedia.org/wiki/Forschungszentrum_J%C3%BClich https://en.wikipedia.org/wiki/Forschungszentrum_J%C3%BClich
- orbifold 3y agoThey used to do nuclear physics research (and still do to some extend) there and had an experimental fast breeder reactor (capable of producing weapons grade plutonium) on campus. They also prepared for rapid development of nuclear weapons capabilities in the 60s. It was a site for a potential German nuclear weapons program, I think they would have been able to produce enough material in ~6 weeks. They frame it as "nuclear disarmament" now of course https://www.fz-juelich.de/en/news/archive/press-release/2022/2022-04-08-nudive-en https://www.fz-juelich.de/en/news/archive/press-release/2022..., but effectively if there is any place where scientists in Germany have active knowledge how to develop nuclear weapons it would be there. There are several former military installations close by, including barracks for a guards and supply company. You don't want something like that in a city centre.
- yvdriess 3y ago> You don't want something like that in a city centre. Nuclear sites also have the tendency to be built on a nation's border.
- ameerhamza8796 3y ago[dead]
- zitterbewegung 3y agoI wonder if we (United States ) could get a risc v in a supercomputer .
- redundantly 3y agoIt's not RISC-V: > SiPearl chose ARM as it is well-established and ready for high-performance applications. Experts say RISC-V is many years away from mainstream server adoption.
- zitterbewegung 3y agoI mean the United States should build a risc v supercomputer and fund the research
- ffgjgf1 3y agoWhy? China would benefit much more than US/Europe from RISC-v catching up. Wouldn’t be the smartest thing to do geopolitically (or longterm economically for that matter).
- moffkalast 3y ago> The Jupiter supercomputer, which will cost €273 million to build, will pair SiPearl’s Rhea processor, which is based on ARM architecture, with accelerator technology from Nvidia. An ARM and a leg, for sure.
- hashtag-til 3y agoThis is really cool, and as it uses an established architecture, can benefit from the software ecosystem that exists around it. The article focuses basically on the x86 vs. arm competition. Any idea where to read more about the application this machine is expected to run? I guess the usual like weather forecast and such?
- owlbite 3y agoIt seems odd to focus on the CPU here, when presumably the vast majority of those flops re coming from the NVIDIA parts? Are the CPUs expected to contribute significant compute, as opposed to marshaling data in/out of the real compute units?
- jefft255 3y agoYes, CPUs are still the main workhorse for many scientific workloads. Sometimes just because the code hasn’t been ported, sometimes because it’s just not something that a GPU can do well.
- londons_explore 3y ago> just because the code hasn’t been ported, Seems stupid to use millions of dollars of supercomputer time just because you can't be bothered to get a few phd students to spend a few months rewriting in CUDA...
- cmdrk 3y agosometimes the code is deeply complex stuff that has accumulated for over 30 years. to _just_ rewrite it in CUDA can be a massive undertaking that could easily produce subtly incorrect results that end up in papers could propagate far into the future by way of citations etc
- throwaway10965 3y agoSounds like a great job for LLMs. Are there any public repositories of this code? I want to try.
- mlyle 3y agoSounds like a -terrible- job for LLMs, because this is all about attention to detail. Order of operations and specific constructs of how floating point work in the codes in question are usually critical. Have fun: https://www.qsl.net/m5aiq/nec-code/nec2-1.2.1.2.f https://www.qsl.net/m5aiq/nec-code/nec2-1.2.1.2.f
- dopa42365 3y agoIsn't it mostly GPUs/accelerators based rather than CPUs? Does NVIDIA even sell those anymore without the whole package deal since they came up with Grace? The last supercomputer with NVIDIA GPUs and third party CPUs I remember reading about was with Zen 2 cores, multiple years ago.
- dotnet00 3y agoPerlmutter (#8) was just commissioned into full service earlier this year and uses Zen 3 cores with A100s. Leonardo (#4) is also current year and uses Xeon CPUs with A100s. Google's H3 also seems to pair H100s with Xeon CPUs. But yes, the CPU is mostly just a footnote, most of the FLOPs come from the GPUs. Although of course the CPUs still need to be sufficiently fast enough that the GPUs can be kept fed.
- janwas 3y agoGolly. There are probably already enough comments in other threads saying this, but I'm curious how you would justify the description 'footnote'.
- dotnet00 3y agoFootnote relative to the overall FLOPs. IIRC, on Perlmutter's GPU partition, 60 of its 64PFLOPs are represented by the GPUs, with the remaing 4PFLOPs, being the CPUs. In comparison, their previous system Cori, had ~3PFLOPs in the Haswell partition and ~30PFLOPs on the KNL partition. That, to me, indicates that CPU performance is mostly a footnote when GPUs are involved, as anyone who had previously been using Cori and is now on Perlmutter, will not see as dramatic of an improvement if restricted to CPUs but they would if able to use GPUs.
- janwas 3y agoOK, fair :) Do you have any data on how many tasks are actually compute-bound? From where I sit, we're often limited by memory bandwidth. When a CPU such as A64FX or even M2 shows up with decent bandwidth, lo and behold, they are often competitive. I do not understand why we didn't see something like SPR Max years ago.
- znpy 3y agoAs an european... On one hand, it's nice to see funding to european companies to develop european technolgy, aiming at a technological sovereignty. On the other hand, SiPearl looks like it was virtually unknown up to this point, and I can't seem to find anything looking like a cpu review (their website claims they have already released at least one generation of Rhea cpus). So this amount of money might not be wasted but still not optimally spent. Which isn't 100% bad, but at least bittersweet. If anything, without reviews and performance benchmarks, we might just get ExaFLOPS on paper. Like how does one of these Rhea CPUs compare to, say, a Graviton 2/3 or to an Ampere Altra cpu?
- cherryteastain 3y agoThere's only one way to promote domestic industries in this space when they are behind/nonexistent: tons of subsidies, even if the domestic alternatives are worse. That's how China, Taiwan, Korea and Japan did it.
- brucethemoose2 3y agoActually Rhea was "known" for awhile, but reading between the lines, looks like it got delayed and updated: https://www.anandtech.com/show/16072/sipearl-lets-rhea-design-leak-72x-zeus-cores-4x-hbm2e-46-ddr5 https://www.anandtech.com/show/16072/sipearl-lets-rhea-desig... (Neoverse V1 and HBM2e woild make this chip kinda old when its finally operational). CPU design takes many years, and this was a HPC only chip, so it doesn't necessarily need to be marketed and paraded around, and the workloads will be totally different than what Graviton processors run.
- nologic01 3y agoThe economics (and politics) of the HPC ecosystem always feels a bit murky and its impact on the broader computing landscape not as much as you would expect. Expensive, one-of-a-kind designs that push the envelope (and pressumably deliver what they are commissioned for) but are floating somewhere above what the rest of the world is using. What would a healthy EU HPC ecosystem look like? At some point there was some excitement about Beowulf clusters [1]. When building a new supercomputer, think, for example about making at least its main compute units more widely available (universities, startups, SME's etc). HPC Computing is arcane and to tap its potential in the post-Moore's law era it needs to get much more democratized and popular. [1] https://en.wikipedia.org/wiki/Beowulf_cluster https://en.wikipedia.org/wiki/Beowulf_cluster
- AdamN 3y agoThere is also the benefit of all the people that get that experience and then bring it to trad-computing.
- Someone 3y ago> making at least its main compute units more widely available If they did, would anybody want them? Are those units competitive for smaller setups and the kind of jobs they run?
- KeplerBoy 3y agoSure.. Plenty of Supercomputers are just A100s, which are also perfectly usable in a single DL workstation. * at least if they used pci-e
- nologic01 3y agoit would depend heavily on costs and tangible benefits versus e.g., renting something from cloud providers. its a new vista, its anybody's guess how things will look in five years, but when Intel's CEO is touting the era of the "AI PC" [1] their projection must be that a certain market will form around compute intensive local computing (largely prompted by the popularity of LLM/AI but that just one domain). on the second branch of your question, indeed a local "supercomputer piece" should have a sufficient number of CPU/GPU's to pack meaningful computational power. this way it would also require and enable the right kind of tooling and programming that scales to larger sizes. given that algorithms can enhance practically any existing application (productivity, games etc), this might be a case of "build it and they will come" [1] https://www.pcmag.com/news/intel-ceo-get-ready-for-the-ai-pc https://www.pcmag.com/news/intel-ceo-get-ready-for-the-ai-pc
- panick21_ 3y agoI much rather have some investment into a advanced open-source RISC-V CPU and phones and laptops based on that. An Open hardware graphics accilerator would be great too. But if we are gone do a HPC thing, at least make the processor open-hardware and RISC-V.
- brucethemoose2 3y agoThe libraries and compiler infrastructure are not ready. There is even some HPC optimization difficulty with ARM, which is much more mature. Doubly so on the consumer side of things.
- panick21_ 3y agoGood oppertunity to get those things up to speed.
- brucethemoose2 3y agoAnd that is happening right now. RISC-V is coming, it just takes a long time.
- datameta 3y agoA bit sad that while POWER9 processors were used in then-SoTA supercomputers, as far as I can tell POWER10 (which I worked on more) is not being used for scientific/industrial HPC.
- chasil 3y agoI understand that POWER9 was much more open than its successor. Is that a factor?
- wmf 3y agoNo, I think IBM just gave up on HPC.
- stonogo 3y agoCan confirm; supercomputers don't slot neatly enough into quartly EPS goals.
- datameta 3y agoI can see the humor, but the thing is server and mainframe sales already fluctuate based on hardware generation cycles (~3 yrs start to finish, sometimes server overlapping with mainframe or memory controller).
- stonogo 3y agoThose contracts are reliable in that the customer is extremely unlikely to move to a different product line. Especially when you've got a customer locked in, refresh timescales are pretty predictable. HPC contracts are generally borne of federal-agency RFPs, and are extremely competitive, and they only 'pay out' upon a passed acceptance test, so it's not trivially possible to predict which quarter your revenue will land for a given sale. You wind up with sales teams putting tons of work into a contract that didn't get selected, which sucks, but even if you win you might wind up missing sales goals, and then overshooting the mark the following quarter. In a company less hidebound this obviously wouldn't be a problem, but IBM has been run by the beancounters for long enough that the prestige isn't worth the murky forecast.
- riedel 3y ago>The Jülich Supercomputing Centre, which is near Munich, will host the system. Interesting take on geography . The confusion, pobably has to do with the fact that the German tier 0 Gauss super computing center is actually spread over 3 sites (Jülich near Cologne/Aachen, Stuttgart and Garching near Munich)
- tremon 3y agoStuttgart and Garching near Munich This reads weird. It took me way too many seconds of wondering "wouldn't Stuttgart be nearer to... Stuttgart?" before I understood what you wrote. Sometimes the Oxford Comma has value, it seems.
- suyjuris 3y agoThe name of the city is “Garching bei München” which translates to “Garching near Munich”. This disambiguates it from „Garching an der Alz“. (Although Jülich is just called Jülich.)
- brucethemoose2 3y agoSiPearl didn't come out of the blue. This has been in planning for years: https://www.anandtech.com/show/16072/sipearl-lets-rhea-design-leak-72x-zeus-cores-4x-hbm2e-46-ddr5 https://www.anandtech.com/show/16072/sipearl-lets-rhea-desig... https://semiengineering.com/tag/sipearl/ https://semiengineering.com/tag/sipearl/ ...It may even be behind schedule?
- riedel 3y agoInteresting thing is that they do not have a Wikipedia entry. It seems purely a product of JU / European Processor Initiative. If this works it would be one of the few real successes of the European funding framework.
- brucethemoose2 3y agoMany interesting chips escape Wikipedia's gaze. For instance, there was a very interesting x86, Zen-like CPU from Centaur with an onboard AI accelerator that is basically undocumented on Wikipedia: https://fuse.wikichip.org/news/3256/centaur-new-x86-server-processor-packs-an-ai-punch/ https://fuse.wikichip.org/news/3256/centaur-new-x86-server-p... https://fuse.wikichip.org/news/3099/centaur-unveils-its-new-server-class-x86-core-cns-adds-avx-512/ https://fuse.wikichip.org/news/3099/centaur-unveils-its-new-...
- my123 3y agoAt this point, Centaur pretty much no longer exists, with engineers transferred to Intel. CNS never ended up becoming a product.
- brucethemoose2 3y agoYes, but it was still fascinating! Imagine if it came out today. I feel like its the near perfect architecture for cheap GenAI.
- 3y ago
- ThinkBeat 3y agoI get the impression this is more subsidizing domestic development than a decision made for the overall "best" CPU. https://www.eenewseurope.com/en/sipearl-raises-e90m-for-rhea-european-supercomputer-chip/ https://www.eenewseurope.com/en/sipearl-raises-e90m-for-rhea...
- chasil 3y agoHowever, the fact that Fujitsu previously claimed the top-performing "Fugaku" supercomputer with their own custom 48-core CPU (fabbed at TSMC) certainly justifies the choice of an AArch64 design. https://en.wikipedia.org/wiki/Fujitsu_A64FX https://en.wikipedia.org/wiki/Fujitsu_A64FX
- yvdriess 3y agoSure, but they are still going for the (French) homegrown AArch64 solution instead of the Fujitsu design.
- bigdave42 3y agoARM is a UK company - the UK is (stupidly) not part of the EU
- b3orn 3y agoThey chose an ARM CPU by SiPearl. The European Processor Initiative (EPI) lists them as French/German. Interestingly Forschungszentrum Jülich (which this supercomputer is for) is also listed as member of EPI.
- sproketboy 3y ago[dead]
- bee_rider 3y agoI thought SoftBank bought them, making them a Japanese company? In any case, SiPearl seems to be the one designing the actual chip, they are French.
- Havoc 3y agoIf they only spent half the budget then is the Nvidia part still to come?
- wslh 3y agoWhat is the approximate price of the SiPearl’s Rhea processor?
- swores 3y agoI believe it's not announced yet. I'll chuck out an unqualified estimate of €10k each, will find out next year (probably) if I'm anywhere close!