5 ms·

Off topic but "Break the ___ barrier" has got to be my least favorite expression that PR people love. It's a "tell" that an article was written as meaningless p

Off topic but "Break the ___ barrier" has got to be my least favorite expression that PR people love. It's a "tell" that an article was written as meaningless pop science fluff instead of anything serious. The sound barrier is a real physical phenomenon. This is not. There's no barrier! Nothing was broken!

alephnerd 2y ago
The Exascale barrier is an actual barrier in HPC/Distributed Systems. It took 15-20 years to reach this point [0] A lot of innovations in the GPU and distributed ML space were subsidized by this research. Concurrent and Parallel Computing are VERY hard problems. [0] - http://helper.ipam.ucla.edu/publications/nmetut/nmetut_19423.pdf http://helper.ipam.ucla.edu/publications/nmetut/nmetut_19423...
- dghlsakjg 2y ago
  The comment isn't saying that the benchmark isn't useful. They are saying that there is no 'barrier' to be broken. The sound barrier was relevant because there was a significant physical effects to overcome specifically when going trans-sonic. It wasn't a question of just adding more powerful engines to existing aircraft. They didn't choose the sound barrier because its a nice number, it was a big deal because all sorts of things behaved outside of their understanding of aerodynamics at that point. People died in the pursuit of understanding the sound barrier. The 'exascale barrier', afaict, is just another number chosen specifically because it is a round(ish) number. It didn't turn computer scientists into smoking holes in the desert when it went wrong. This is an incremental improvement in an incredible field, but not a world changing watershed moment.
  - trueismywork 2y ago
    It is definitely not incremental. If you watch some talk on Gordon Bell prize for Frontier, you see that the dynamics have changed completely. Data, software stack, I/O have suddenly become bottlenecks in multiple places. So yes, it is a watershed moment..
    - dghlsakjg 2y ago
      Can you explain how 10^18 FLOPS is fundamentally different than (10^18 - 20) FLOPS? Do the conventional rules of computing completely change at that exact number?
      - trueismywork 2y ago
        https://www.osti.gov/servlets/purl/1902810 https://www.osti.gov/servlets/purl/1902810 See this for example. Different applications have different scales at which they reach similar problems. Exascale is decent upper bound for most of the fields. If you really dig in deep, the bound may be found at slightly lower value > 500 Pflops. But it's a good rule of thumb to consider 1 EFlop/s to be safe. Also see this https://irp.fas.org/agency/dod/jason/exascale.pdf https://irp.fas.org/agency/dod/jason/exascale.pdf
    - dekhn 2y ago
      But that same point was made at every 3 levels of magnitude improvement in supercomputing history.
      - trueismywork 2y ago
        For example, In situ visualization is now preferred due I/O bottleneck in application instead of compute bottleneck. This seems different from last generations.
  - alephnerd 2y ago
    When Exascale was defined as a barrier in the mid-late 2000s, a lot of computing technologies and techniques that are taken for granted today did not exist outside of the lab. For example, FPGAs were considered as much more viable for sparse matrix computation instead of GPUs, BLAS implementations were not as robust yet, parallel programming APIs like CUDA and Vulkan were in their infancy, etc. Just because you didn't do well in your systems classes or you think spinning up an EC2 instance on AWS is "easy" doesn't mean it's an easy problem. That's like saying Einstein or Planck are dummies because AP Physics E&M students can handle basic relatively and quantum theory.
    - dghlsakjg 2y ago
      > Just because you didn't do well in your systems classes or you think spinning up an EC2 instance on AWS is "easy" doesn't mean it's an easy problem. Please leave personal attacks out of this, it is not in the spirit of HN, or in helping to see people's perspectives. I'm not saying its not a hard problem, not at all. I respect the hell out of the work that has been done here. I'm saying that the sound barrier is called a barrier for a very good reason. Aerodynamics on one side of the sound barrier are different than aerodynamics on the other. It is a different game entirely. That is why it is considered a barrier. A plane that has superb subsonic aerodynamics will not perform well on the other side of that barrier Exascale computers on the other hand, while truly amazing, are not operating differently by hitting 10^18 FLOPS. If you're computer does 10^18 -20 FLOPS it is not operating in a fundamentally different set of rules than one running above the exaflop benchmark. I never said that the achievement wasn't laudable. I argued that there is no barrier there. If I'm wrong I would appreciate you explaining why doing things at 10^18 FLOPS is fundamentally different than computing just below that benchmark.
      - alephnerd 2y ago
        Because the unit used to measure IO is FLOPS [0], and most societies have settled on base-10 as their numerical system of choice for thousands of years. Furthermore, it is not possible to predict the exact number of cycles you'll need, since that depends on the architecture, compiler and many other factors. This is why FLOPS are used as the unit of choice. Each jump in flops by a magnitude of 10^3 is a significant problem in concurrency, IO, parallelism, storage, and existing compute infrastructure. Managing racks is difficult, managing concurrent workloads is difficult, managing I/O and storage is difficult, designing compute infra like FPGAs/GPUs/CPUs for this is difficult, etc. [0] - https://en.m.wikipedia.org/wiki/FLOPS https://en.m.wikipedia.org/wiki/FLOPS
  - K0balt 2y ago
    “It didn't turn computer scientists into smoking holes in the desert when it went wrong. “ Shame, really, HPC could use a little bit of high stakes adventure to make it sexier. (Funny how risking death makes things more attractive ?) There’s something about working with equipment where the line between top performance and a smoking hole is a matter of degree. Also opens up a lot more Netflix production opportunities and I bet code safety would get a bump as well.
theandrewbailey 2y ago
100% agreed. I might put my thoughts about it into an essay and name it "Breaking barriers considered harmful".

alephnerd 2y ago

The Exascale barrier is an actual barrier in HPC/Distributed Systems. It took 15-20 years to reach this point [0] A lot of innovations in the GPU and distributed ML space were subsidized by this research. Concurrent and Parallel Computing are VERY hard problems. [0] - http://helper.ipam.ucla.edu/publications/nmetut/nmetut_19423.pdf http://helper.ipam.ucla.edu/publications/nmetut/nmetut_19423...

dghlsakjg 2y ago
The comment isn't saying that the benchmark isn't useful. They are saying that there is no 'barrier' to be broken. The sound barrier was relevant because there was a significant physical effects to overcome specifically when going trans-sonic. It wasn't a question of just adding more powerful engines to existing aircraft. They didn't choose the sound barrier because its a nice number, it was a big deal because all sorts of things behaved outside of their understanding of aerodynamics at that point. People died in the pursuit of understanding the sound barrier. The 'exascale barrier', afaict, is just another number chosen specifically because it is a round(ish) number. It didn't turn computer scientists into smoking holes in the desert when it went wrong. This is an incremental improvement in an incredible field, but not a world changing watershed moment.
- trueismywork 2y ago
  It is definitely not incremental. If you watch some talk on Gordon Bell prize for Frontier, you see that the dynamics have changed completely. Data, software stack, I/O have suddenly become bottlenecks in multiple places. So yes, it is a watershed moment..
  - dghlsakjg 2y ago
    Can you explain how 10^18 FLOPS is fundamentally different than (10^18 - 20) FLOPS? Do the conventional rules of computing completely change at that exact number?
    - trueismywork 2y ago
      https://www.osti.gov/servlets/purl/1902810 https://www.osti.gov/servlets/purl/1902810 See this for example. Different applications have different scales at which they reach similar problems. Exascale is decent upper bound for most of the fields. If you really dig in deep, the bound may be found at slightly lower value > 500 Pflops. But it's a good rule of thumb to consider 1 EFlop/s to be safe. Also see this https://irp.fas.org/agency/dod/jason/exascale.pdf https://irp.fas.org/agency/dod/jason/exascale.pdf
  - dekhn 2y ago
    But that same point was made at every 3 levels of magnitude improvement in supercomputing history.
    - trueismywork 2y ago
      For example, In situ visualization is now preferred due I/O bottleneck in application instead of compute bottleneck. This seems different from last generations.
- alephnerd 2y ago
  When Exascale was defined as a barrier in the mid-late 2000s, a lot of computing technologies and techniques that are taken for granted today did not exist outside of the lab. For example, FPGAs were considered as much more viable for sparse matrix computation instead of GPUs, BLAS implementations were not as robust yet, parallel programming APIs like CUDA and Vulkan were in their infancy, etc. Just because you didn't do well in your systems classes or you think spinning up an EC2 instance on AWS is "easy" doesn't mean it's an easy problem. That's like saying Einstein or Planck are dummies because AP Physics E&M students can handle basic relatively and quantum theory.
  - dghlsakjg 2y ago
    > Just because you didn't do well in your systems classes or you think spinning up an EC2 instance on AWS is "easy" doesn't mean it's an easy problem. Please leave personal attacks out of this, it is not in the spirit of HN, or in helping to see people's perspectives. I'm not saying its not a hard problem, not at all. I respect the hell out of the work that has been done here. I'm saying that the sound barrier is called a barrier for a very good reason. Aerodynamics on one side of the sound barrier are different than aerodynamics on the other. It is a different game entirely. That is why it is considered a barrier. A plane that has superb subsonic aerodynamics will not perform well on the other side of that barrier Exascale computers on the other hand, while truly amazing, are not operating differently by hitting 10^18 FLOPS. If you're computer does 10^18 -20 FLOPS it is not operating in a fundamentally different set of rules than one running above the exaflop benchmark. I never said that the achievement wasn't laudable. I argued that there is no barrier there. If I'm wrong I would appreciate you explaining why doing things at 10^18 FLOPS is fundamentally different than computing just below that benchmark.
    - alephnerd 2y ago
      Because the unit used to measure IO is FLOPS [0], and most societies have settled on base-10 as their numerical system of choice for thousands of years. Furthermore, it is not possible to predict the exact number of cycles you'll need, since that depends on the architecture, compiler and many other factors. This is why FLOPS are used as the unit of choice. Each jump in flops by a magnitude of 10^3 is a significant problem in concurrency, IO, parallelism, storage, and existing compute infrastructure. Managing racks is difficult, managing concurrent workloads is difficult, managing I/O and storage is difficult, designing compute infra like FPGAs/GPUs/CPUs for this is difficult, etc. [0] - https://en.m.wikipedia.org/wiki/FLOPS https://en.m.wikipedia.org/wiki/FLOPS
- K0balt 2y ago
  “It didn't turn computer scientists into smoking holes in the desert when it went wrong. “ Shame, really, HPC could use a little bit of high stakes adventure to make it sexier. (Funny how risking death makes things more attractive ?) There’s something about working with equipment where the line between top performance and a smoking hole is a matter of degree. Also opens up a lot more Netflix production opportunities and I bet code safety would get a bump as well.

theandrewbailey 2y ago

100% agreed. I might put my thoughts about it into an essay and name it "Breaking barriers considered harmful".