4 ms·
When Exascale was defined as a barrier in the mid-late 2000s, a lot of computing technologies and techniques that are taken for granted today did not exist outs
by alephnerd 3y ago
When Exascale was defined as a barrier in the mid-late 2000s, a lot of computing technologies and techniques that are taken for granted today did not exist outside of the lab.
For example, FPGAs were considered as much more viable for sparse matrix computation instead of GPUs, BLAS implementations were not as robust yet, parallel programming APIs like CUDA and Vulkan were in their infancy, etc.
Just because you didn't do well in your systems classes or you think spinning up an EC2 instance on AWS is "easy" doesn't mean it's an easy problem.
That's like saying Einstein or Planck are dummies because AP Physics E&M students can handle basic relatively and quantum theory.
- dghlsakjg 3y ago> Just because you didn't do well in your systems classes or you think spinning up an EC2 instance on AWS is "easy" doesn't mean it's an easy problem. Please leave personal attacks out of this, it is not in the spirit of HN, or in helping to see people's perspectives. I'm not saying its not a hard problem, not at all. I respect the hell out of the work that has been done here. I'm saying that the sound barrier is called a barrier for a very good reason. Aerodynamics on one side of the sound barrier are different than aerodynamics on the other. It is a different game entirely. That is why it is considered a barrier. A plane that has superb subsonic aerodynamics will not perform well on the other side of that barrier Exascale computers on the other hand, while truly amazing, are not operating differently by hitting 10^18 FLOPS. If you're computer does 10^18 -20 FLOPS it is not operating in a fundamentally different set of rules than one running above the exaflop benchmark. I never said that the achievement wasn't laudable. I argued that there is no barrier there. If I'm wrong I would appreciate you explaining why doing things at 10^18 FLOPS is fundamentally different than computing just below that benchmark.
- alephnerd 3y agoBecause the unit used to measure IO is FLOPS [0], and most societies have settled on base-10 as their numerical system of choice for thousands of years. Furthermore, it is not possible to predict the exact number of cycles you'll need, since that depends on the architecture, compiler and many other factors. This is why FLOPS are used as the unit of choice. Each jump in flops by a magnitude of 10^3 is a significant problem in concurrency, IO, parallelism, storage, and existing compute infrastructure. Managing racks is difficult, managing concurrent workloads is difficult, managing I/O and storage is difficult, designing compute infra like FPGAs/GPUs/CPUs for this is difficult, etc. [0] - https://en.m.wikipedia.org/wiki/FLOPS https://en.m.wikipedia.org/wiki/FLOPS
- dghlsakjg 3y agoSo its just an arbitrary base-10 number, and not a constraint imposed by physics or some other outside constraint like the sound barrier? That's kind of my point, an Exaflop is a benchmark and not a 'barrier'. The sound barrier wasn't a 10^3 change in speed, the difference between a subsonic plane and a supersonic plane is measured in percentages when it comes to speed, e.g. a plane that is happy at .8 Mach for a top speed is going to be designed under a completely different set of rules than one that tops out at 1.2 Mach. Again, I'm not saying that the accomplishments are insignificant, I'm just arguing press-release semantics here.
- alephnerd 3y agoIf you're not being facetious I recommend listening to the 2022 ACM Gordon Bell Award Winner lecture. What you're doing is the equivalent of asking why do we use Gigabyte or Terabyte as a metric. Just reaching 10^12 floating point operations per second was not something that was done until 2018.
- kortilla 3y agoNo, it’s not equivalent at all. Nobody is asking about the unit of measure.
- dekhn 3y agoExascale was an arbitrary target- it didn't unlock any magical capabilities. In fact the supercomputer folks are now saying the real barrier that will unlock their science is "zettaflops" (just saw a post from a luminary about it). (also please don't be rude or condescending, it detracts from your argument)
- alephnerd 3y agoExascale is the primary target decided among everyone in the HPC community in the mid-late 2000s because FLOPS (floating point operations per second) is the unit used to benchmark, as there are various different variables like compiler, architecture, etc are very difficult to account for. It's functionally the same as arguing that GB or TB are arbitrary units to represent storage.
- dekhn 3y agoYes, that's correct: GB and TB are arbitrary units. We use them because historically, successive multiples of 1000 have been used to represent significant changes in size, mass, and velocity. I used to work with/for NERSC and was around when they announced exascale as a target, and now they want to target zettascale. There is no magic threshold where science simulations suddenly work better as you scale up. It's mainly about setting goals that are 10-15 years away to stimulate spending and research.
- alephnerd 3y ago> I used to work with/for NERSC and was around when they announced exascale as a target We most likely crossed paths. How close were you to faculty in AMPLab? > It's mainly about setting goals that are 10-15 years away to stimulate spending and research. And that's not a barrier to you? That feels very condescending about applied research or the amount of effort put into the entire Distributed Systems field.
- dekhn 3y agoI am being condescending about the effort put into the distributed systems field- very specifically, about classical supercomputers. How close was I to faculty in AMPLab? Pretty close; I attended a retreat one year, helped steer funding their way, tried to hire Matei into Google Research, and have chatted with Patterson extensively around the time he wrote this (https://www.nytimes.com/2011/12/06/science/david-patterson-enlist-computer-scientists-in-cancer-fight.html https://www.nytimes.com/2011/12/06/science/david-patterson-e...) then later when he worked on TPUs at Google. (I'm not a stellar researcher or anything, really more of a functionary, but the one thing I do have is an absolutely realistic understanding of how academic and industrial HPC works)
- marshray 3y agoWhat specifically about the system changes as you cross from "almost exascale" to "definitely exascale" to justify calling it a 'barrier'?
- alephnerd 3y agoThe same reason we choose to define a Gigabyte as 10^9 or use the Richter scale to measure earthquakes. We need a benchmark to delineate between large magnitudes. Furthermore, there are very real engineering problems that had to be solved to even reach this point. A lot of noobs take GPUs, Scipy, BLAS, CUDA, etc for granted when in reality all of these were subsidized by HPC research. The DoE's Exascale project was one of the largest buyers for compute for much of the 21st century and helped subsidize Nvidia, Intel, AMD, and other vendors when they were at their worst financially.
- marshray 3y agoDid you even read the question?
- alephnerd 3y agoAnd I answered - building the logistics and ecosystem
- Dalewyn 3y agoHe asked what is significant about 1 exascale that it's a "barrier". Now granted, the original rendition of this saying ("Breaking the sound barrier.") is also arbitrary because mach 1 is the speed of sound travelling through air on planet Earth, it's still a valid question that you did not answer.
- dghlsakjg 3y agoBreaking the sound barrier isn't arbitrary. Indeed even measuring it using the Mach number shows that. Mach 1 isn't fixed, it is variable based on a number of attributes. At Mach numbers above 1 the compressibility of the air is entirely different. The medium in which the airplane operates behaves differently, in other words. The sound barrier was a barrier because the planes they were using stopped behaving predictably at Mach > 1. They had to learn to design planes differently if they wanted to fly at those speeds. Mach 1 is an external constraint mandated by the laws of physics. There is a good reason that sound can't travel faster. That is why it is a barrier to be broken. It is a paradigm shift imposed entirely by the properties of our physical world.