4 ms·

I think that with the end of Moore's law, there is a gradual unwinding of software bloat and inefficiency. Once the easy hardware gains die away, it is worth st

I think that with the end of Moore's law, there is a gradual unwinding of software bloat and inefficiency. Once the easy hardware gains die away, it is worth studying how software can be better, but it's a cultural transition that takes time.

stephc_int13 6y ago
I also think that the end of the so called Moore Law will create the necessary incentives to build better software.
- ginko 6y ago
  What's with the "so called"? Moore's Law is a pretty well-established term.
  - stephc_int13 6y ago
    It is not a law, it is merely an observation.
jhayward 6y ago
It is also the case that in some/many areas software has improved performance by several multiples of the improvement in hardware performance over the last 20 years. So it makes sense that this would invigorate software performance investment.
- mpweiher 6y ago
  Can you give some examples of areas where software performance improvement has been several multiples of hardware improvement? Most of the examples I can think of are ones where the software slowdown has more than cancelled out the hardware improvements. Then there are a some areas where hardware performance improvement was sufficient to overcome software slowdown. Software getting faster? Software getting faster than hardware??
  - MayeulC 6y ago
    Compilers are way smarter and can make old code faster than it used to run. They also parse the code way faster, and would as such compile faster if you restrict them to the level of optimizations they used to do. "Interpreters" are faster as well: Lisp runs faster, JavaScript is orders of magnitudes more efficient as well. Algorithms have been refined. Faster paths have been uncovered for matrix multiplication [1] (unsure if the latest improvements are leveraged) and other algorithms. Use-cases that have been around fo run a wile (say, h.264 encode/decode) are more optimized. We now tend to be a lot better at managing concurrency too (see: rust, openmp and others), with the massively parallel architectures that come out nowadays. [1] https://en.m.wikipedia.org/wiki/Matrix_multiplication_algorithm https://en.m.wikipedia.org/wiki/Matrix_multiplication_algori...
    - mpweiher 6y ago
      Thanks for the examples! However, I can't really agree that they are examples of software improvements being multiples of hardware improvements. 1. Compiler optimizations See Proebsting's Law [1], which states that whereas Moore's Law provided a doubling of performance ever 18-24 months, compiler optimizations provide a doubling every 18 years at best. More recent measurements indicate that this was optimistic. 2. Compilers getting faster Sorry, not seeing it. Swift, for example, can take a minute before giving up on a one line expression, and has been clocked at 16 lines/second for some codebases. All the while producing slow code. See also part of the motivation for Jonathan Blow's Jai programming language. 3. Matrix multiplication No numbers given, so ¯\_(ツ)_/¯ 4. h.264/h.265 The big improvements have come from moving them to hardware. 5. Concurrency That's hardware. [1] https://www.semanticscholar.org/paper/On-Proebsting%27%27s-Law-Scott/0a2b1aa8bb63fb545f7f41233e5d6c0206486ccc?p2df https://www.semanticscholar.org/paper/On-Proebsting%27%27s-L...
      - MayeulC 6y ago
        You are reading my examples in bad faith :) (though I originally missed your point about "multiples of") You want examples where software has sped up at a rate faster than hardware (meaning that new software on old hardware runs faster than old software on new hardware). Javascript might not have been a good idea in the first place, but I bet that if you were to run V8 (if you have enough RAM) on 2005-era commodity hardware, it would be faster than running 2005-SpiderMonkey on today's hardware. JIT compilers have improved (including lisps, php, python, etc). Can you give me an example of 2005-era swift running faster on newer hardware than today's compiler on yesterday's hardware? You can't, as this is a new language, with new semantics and possibilities. Parsing isn't as simple as it seems, you can't really compare two different languages. These software improvements also tend to pile up along the stack. And comparing HW to SW is tricky: you can always cram more HW to gain more performance, while using more SW unfortunately tends to have the opposite effect. So you have to restrict yourself HW-wise: same price? same power requirements? I'd tend to go with the latter as HW has enjoyed economies of scale SW can't. Concurrency might be hardware, but in keeping with the above point, more execution cores will be useless for a multithread-unaware program. Old software might not run better on new HW, but old HW didn't have these capabilities, so the opposite is probably true as well. Keep in mind that these new HW developments were enabled by SW developments. > No numbers given, so ¯\_(ツ)_/¯ Big-O notation should speak for itself, I am not going to try and resurrect a BLAS package from the 80s to benchmark against on a PIC just for this argument ;) Other noteworthy algorithms include the FFT [1]. (I had another one in mind but lost it). > The big improvements have come from moving them to hardware. I'm talking specifically about SW implementations. Of course you can design an ASIC for most stuff. And most performance-critical applications probably had ASICs designed for them by now, helping prove your point. SW and HW are not isolated either, and an algorithm optimized for old HW might be extremely inneficient on new HW, and vice-versa. And in any case, HW developments were in large part enabled by SW developments with logic synthesis, place and route, etc. HW development is SW development to a large extent today, though that was not your original point. What can't be argued against, however, is that both SW and HW improvements have made it much easier to create both HW and SW. Whether SW or HW has been most instrumental with this, I am not sure. They are tightly coupled: it's much easier to write a complex program with a modern compiler, but would you wait for it to compile on an old machine? Likewise for logic synthesis tools and HW simulators. Low-effort development can get you further, and that shows. I guess that's what you are complaining about. [1] https://en.wikipedia.org/wiki/Fast_Fourier_transform#Algorithms https://en.wikipedia.org/wiki/Fast_Fourier_transform#Algorit...

stephc_int13 6y ago

I also think that the end of the so called Moore Law will create the necessary incentives to build better software.

ginko 6y ago
What's with the "so called"? Moore's Law is a pretty well-established term.
- stephc_int13 6y ago
  It is not a law, it is merely an observation.

jhayward 6y ago

It is also the case that in some/many areas software has improved performance by several multiples of the improvement in hardware performance over the last 20 years. So it makes sense that this would invigorate software performance investment.

mpweiher 6y ago
Can you give some examples of areas where software performance improvement has been several multiples of hardware improvement? Most of the examples I can think of are ones where the software slowdown has more than cancelled out the hardware improvements. Then there are a some areas where hardware performance improvement was sufficient to overcome software slowdown. Software getting faster? Software getting faster than hardware??
- MayeulC 6y ago
  Compilers are way smarter and can make old code faster than it used to run. They also parse the code way faster, and would as such compile faster if you restrict them to the level of optimizations they used to do. "Interpreters" are faster as well: Lisp runs faster, JavaScript is orders of magnitudes more efficient as well. Algorithms have been refined. Faster paths have been uncovered for matrix multiplication [1] (unsure if the latest improvements are leveraged) and other algorithms. Use-cases that have been around fo run a wile (say, h.264 encode/decode) are more optimized. We now tend to be a lot better at managing concurrency too (see: rust, openmp and others), with the massively parallel architectures that come out nowadays. [1] https://en.m.wikipedia.org/wiki/Matrix_multiplication_algorithm https://en.m.wikipedia.org/wiki/Matrix_multiplication_algori...
  - mpweiher 6y ago
    Thanks for the examples! However, I can't really agree that they are examples of software improvements being multiples of hardware improvements. 1. Compiler optimizations See Proebsting's Law [1], which states that whereas Moore's Law provided a doubling of performance ever 18-24 months, compiler optimizations provide a doubling every 18 years at best. More recent measurements indicate that this was optimistic. 2. Compilers getting faster Sorry, not seeing it. Swift, for example, can take a minute before giving up on a one line expression, and has been clocked at 16 lines/second for some codebases. All the while producing slow code. See also part of the motivation for Jonathan Blow's Jai programming language. 3. Matrix multiplication No numbers given, so ¯\_(ツ)_/¯ 4. h.264/h.265 The big improvements have come from moving them to hardware. 5. Concurrency That's hardware. [1] https://www.semanticscholar.org/paper/On-Proebsting%27%27s-Law-Scott/0a2b1aa8bb63fb545f7f41233e5d6c0206486ccc?p2df https://www.semanticscholar.org/paper/On-Proebsting%27%27s-L...
    - MayeulC 6y ago
      You are reading my examples in bad faith :) (though I originally missed your point about "multiples of") You want examples where software has sped up at a rate faster than hardware (meaning that new software on old hardware runs faster than old software on new hardware). Javascript might not have been a good idea in the first place, but I bet that if you were to run V8 (if you have enough RAM) on 2005-era commodity hardware, it would be faster than running 2005-SpiderMonkey on today's hardware. JIT compilers have improved (including lisps, php, python, etc). Can you give me an example of 2005-era swift running faster on newer hardware than today's compiler on yesterday's hardware? You can't, as this is a new language, with new semantics and possibilities. Parsing isn't as simple as it seems, you can't really compare two different languages. These software improvements also tend to pile up along the stack. And comparing HW to SW is tricky: you can always cram more HW to gain more performance, while using more SW unfortunately tends to have the opposite effect. So you have to restrict yourself HW-wise: same price? same power requirements? I'd tend to go with the latter as HW has enjoyed economies of scale SW can't. Concurrency might be hardware, but in keeping with the above point, more execution cores will be useless for a multithread-unaware program. Old software might not run better on new HW, but old HW didn't have these capabilities, so the opposite is probably true as well. Keep in mind that these new HW developments were enabled by SW developments. > No numbers given, so ¯\_(ツ)_/¯ Big-O notation should speak for itself, I am not going to try and resurrect a BLAS package from the 80s to benchmark against on a PIC just for this argument ;) Other noteworthy algorithms include the FFT [1]. (I had another one in mind but lost it). > The big improvements have come from moving them to hardware. I'm talking specifically about SW implementations. Of course you can design an ASIC for most stuff. And most performance-critical applications probably had ASICs designed for them by now, helping prove your point. SW and HW are not isolated either, and an algorithm optimized for old HW might be extremely inneficient on new HW, and vice-versa. And in any case, HW developments were in large part enabled by SW developments with logic synthesis, place and route, etc. HW development is SW development to a large extent today, though that was not your original point. What can't be argued against, however, is that both SW and HW improvements have made it much easier to create both HW and SW. Whether SW or HW has been most instrumental with this, I am not sure. They are tightly coupled: it's much easier to write a complex program with a modern compiler, but would you wait for it to compile on an old machine? Likewise for logic synthesis tools and HW simulators. Low-effort development can get you further, and that shows. I guess that's what you are complaining about. [1] https://en.wikipedia.org/wiki/Fast_Fourier_transform#Algorithms https://en.wikipedia.org/wiki/Fast_Fourier_transform#Algorit...