5 ms·

It is also the case that in some/many areas software has improved performance by several multiples of the improvement in hardware performance over the last 20 y

It is also the case that in some/many areas software has improved performance by several multiples of the improvement in hardware performance over the last 20 years. So it makes sense that this would invigorate software performance investment.

mpweiher 6y ago
Can you give some examples of areas where software performance improvement has been several multiples of hardware improvement? Most of the examples I can think of are ones where the software slowdown has more than cancelled out the hardware improvements. Then there are a some areas where hardware performance improvement was sufficient to overcome software slowdown. Software getting faster? Software getting faster than hardware??
- MayeulC 6y ago
  Compilers are way smarter and can make old code faster than it used to run. They also parse the code way faster, and would as such compile faster if you restrict them to the level of optimizations they used to do. "Interpreters" are faster as well: Lisp runs faster, JavaScript is orders of magnitudes more efficient as well. Algorithms have been refined. Faster paths have been uncovered for matrix multiplication [1] (unsure if the latest improvements are leveraged) and other algorithms. Use-cases that have been around fo run a wile (say, h.264 encode/decode) are more optimized. We now tend to be a lot better at managing concurrency too (see: rust, openmp and others), with the massively parallel architectures that come out nowadays. [1] https://en.m.wikipedia.org/wiki/Matrix_multiplication_algorithm https://en.m.wikipedia.org/wiki/Matrix_multiplication_algori...
  - mpweiher 6y ago
    Thanks for the examples! However, I can't really agree that they are examples of software improvements being multiples of hardware improvements. 1. Compiler optimizations See Proebsting's Law [1], which states that whereas Moore's Law provided a doubling of performance ever 18-24 months, compiler optimizations provide a doubling every 18 years at best. More recent measurements indicate that this was optimistic. 2. Compilers getting faster Sorry, not seeing it. Swift, for example, can take a minute before giving up on a one line expression, and has been clocked at 16 lines/second for some codebases. All the while producing slow code. See also part of the motivation for Jonathan Blow's Jai programming language. 3. Matrix multiplication No numbers given, so ¯\_(ツ)_/¯ 4. h.264/h.265 The big improvements have come from moving them to hardware. 5. Concurrency That's hardware. [1] https://www.semanticscholar.org/paper/On-Proebsting%27%27s-Law-Scott/0a2b1aa8bb63fb545f7f41233e5d6c0206486ccc?p2df https://www.semanticscholar.org/paper/On-Proebsting%27%27s-L...
    - MayeulC 6y ago
      You are reading my examples in bad faith :) (though I originally missed your point about "multiples of") You want examples where software has sped up at a rate faster than hardware (meaning that new software on old hardware runs faster than old software on new hardware). Javascript might not have been a good idea in the first place, but I bet that if you were to run V8 (if you have enough RAM) on 2005-era commodity hardware, it would be faster than running 2005-SpiderMonkey on today's hardware. JIT compilers have improved (including lisps, php, python, etc). Can you give me an example of 2005-era swift running faster on newer hardware than today's compiler on yesterday's hardware? You can't, as this is a new language, with new semantics and possibilities. Parsing isn't as simple as it seems, you can't really compare two different languages. These software improvements also tend to pile up along the stack. And comparing HW to SW is tricky: you can always cram more HW to gain more performance, while using more SW unfortunately tends to have the opposite effect. So you have to restrict yourself HW-wise: same price? same power requirements? I'd tend to go with the latter as HW has enjoyed economies of scale SW can't. Concurrency might be hardware, but in keeping with the above point, more execution cores will be useless for a multithread-unaware program. Old software might not run better on new HW, but old HW didn't have these capabilities, so the opposite is probably true as well. Keep in mind that these new HW developments were enabled by SW developments. > No numbers given, so ¯\_(ツ)_/¯ Big-O notation should speak for itself, I am not going to try and resurrect a BLAS package from the 80s to benchmark against on a PIC just for this argument ;) Other noteworthy algorithms include the FFT [1]. (I had another one in mind but lost it). > The big improvements have come from moving them to hardware. I'm talking specifically about SW implementations. Of course you can design an ASIC for most stuff. And most performance-critical applications probably had ASICs designed for them by now, helping prove your point. SW and HW are not isolated either, and an algorithm optimized for old HW might be extremely inneficient on new HW, and vice-versa. And in any case, HW developments were in large part enabled by SW developments with logic synthesis, place and route, etc. HW development is SW development to a large extent today, though that was not your original point. What can't be argued against, however, is that both SW and HW improvements have made it much easier to create both HW and SW. Whether SW or HW has been most instrumental with this, I am not sure. They are tightly coupled: it's much easier to write a complex program with a modern compiler, but would you wait for it to compile on an old machine? Likewise for logic synthesis tools and HW simulators. Low-effort development can get you further, and that shows. I guess that's what you are complaining about. [1] https://en.wikipedia.org/wiki/Fast_Fourier_transform#Algorithms https://en.wikipedia.org/wiki/Fast_Fourier_transform#Algorit...
- jhayward 6y ago
  I think a better answer than I could give can be found at [1]. One pull quote they highlight: But the White House advisory report cited research, including a study of progress over a 15-year span on a benchmark production-planning task. Over that time, the speed of completing the calculations improved by a factor of 43 million. Of the total, a factor of roughly 1,000 was attributable to faster processor speeds, according to the research by Martin Grotschel, a German scientist and mathematician. Yet a factor of 43,000 was due to improvements in the efficiency of software algorithms. [1] https://cstheory.stackexchange.com/questions/12905/speedup-from-algorithmic-advances-vs-hardware https://cstheory.stackexchange.com/questions/12905/speedup-f...
  - mpweiher 6y ago
    Fun example, thanks for pointing it out! I actually took Professor Grötschel's Linear Optimization course at TU Berlin, and the practical optimization task/competition we did for that course very much illustrates the point made in the answer to the stackexchange question you posted. Our team won the competition, beating the performance not just of the other student teams' programs, but also the program of the professor's assistants, by around an order of magnitude. How? By changing a single "<" (less than) to "<=" (less than or equal), which dramatically reduced the run-time of the dominant problem of the problem-set. This really miffed the professor quite a bit, because we were just a bunch of dumb CS majors taking a class in the far superior math department, but he was a good sport about it and we got a nice little prize in addition to our grade. It also helped that our program was still fastest without that one change, though now with only a tiny margin. The point being that, as the post notes, this is a single problem in a single, very specialized discipline, and this example absolutely does not generalize.
    - jhayward 6y ago
      I think that your original comment is the one that risks over-generalization. The "software gets slower" perception is in a very narrow technical niche: user interface/user experience for consumer applications. And it has a common cause: product managers and developers stuff as much in the the user experience as they can until performance suffers. But even within that over-stuffing phenomenon you can see individual components that have the same software improvement curve. In any area where computing performance has been critical - optimization, data management, simulation, physical modeling, statistical analysis, weather forecasting, genomics, computational medicine, imagery, etc. there have been many, many cases of software outpacing hardware in rate of improvement. Enough so that it is normal to expect it, and something to investigate for cause if it's not seen.

mpweiher 6y ago

Can you give some examples of areas where software performance improvement has been several multiples of hardware improvement? Most of the examples I can think of are ones where the software slowdown has more than cancelled out the hardware improvements. Then there are a some areas where hardware performance improvement was sufficient to overcome software slowdown. Software getting faster? Software getting faster than hardware??

MayeulC 6y ago
Compilers are way smarter and can make old code faster than it used to run. They also parse the code way faster, and would as such compile faster if you restrict them to the level of optimizations they used to do. "Interpreters" are faster as well: Lisp runs faster, JavaScript is orders of magnitudes more efficient as well. Algorithms have been refined. Faster paths have been uncovered for matrix multiplication [1] (unsure if the latest improvements are leveraged) and other algorithms. Use-cases that have been around fo run a wile (say, h.264 encode/decode) are more optimized. We now tend to be a lot better at managing concurrency too (see: rust, openmp and others), with the massively parallel architectures that come out nowadays. [1] https://en.m.wikipedia.org/wiki/Matrix_multiplication_algorithm https://en.m.wikipedia.org/wiki/Matrix_multiplication_algori...
- mpweiher 6y ago
  Thanks for the examples! However, I can't really agree that they are examples of software improvements being multiples of hardware improvements. 1. Compiler optimizations See Proebsting's Law [1], which states that whereas Moore's Law provided a doubling of performance ever 18-24 months, compiler optimizations provide a doubling every 18 years at best. More recent measurements indicate that this was optimistic. 2. Compilers getting faster Sorry, not seeing it. Swift, for example, can take a minute before giving up on a one line expression, and has been clocked at 16 lines/second for some codebases. All the while producing slow code. See also part of the motivation for Jonathan Blow's Jai programming language. 3. Matrix multiplication No numbers given, so ¯\_(ツ)_/¯ 4. h.264/h.265 The big improvements have come from moving them to hardware. 5. Concurrency That's hardware. [1] https://www.semanticscholar.org/paper/On-Proebsting%27%27s-Law-Scott/0a2b1aa8bb63fb545f7f41233e5d6c0206486ccc?p2df https://www.semanticscholar.org/paper/On-Proebsting%27%27s-L...
  - MayeulC 6y ago
    You are reading my examples in bad faith :) (though I originally missed your point about "multiples of") You want examples where software has sped up at a rate faster than hardware (meaning that new software on old hardware runs faster than old software on new hardware). Javascript might not have been a good idea in the first place, but I bet that if you were to run V8 (if you have enough RAM) on 2005-era commodity hardware, it would be faster than running 2005-SpiderMonkey on today's hardware. JIT compilers have improved (including lisps, php, python, etc). Can you give me an example of 2005-era swift running faster on newer hardware than today's compiler on yesterday's hardware? You can't, as this is a new language, with new semantics and possibilities. Parsing isn't as simple as it seems, you can't really compare two different languages. These software improvements also tend to pile up along the stack. And comparing HW to SW is tricky: you can always cram more HW to gain more performance, while using more SW unfortunately tends to have the opposite effect. So you have to restrict yourself HW-wise: same price? same power requirements? I'd tend to go with the latter as HW has enjoyed economies of scale SW can't. Concurrency might be hardware, but in keeping with the above point, more execution cores will be useless for a multithread-unaware program. Old software might not run better on new HW, but old HW didn't have these capabilities, so the opposite is probably true as well. Keep in mind that these new HW developments were enabled by SW developments. > No numbers given, so ¯\_(ツ)_/¯ Big-O notation should speak for itself, I am not going to try and resurrect a BLAS package from the 80s to benchmark against on a PIC just for this argument ;) Other noteworthy algorithms include the FFT [1]. (I had another one in mind but lost it). > The big improvements have come from moving them to hardware. I'm talking specifically about SW implementations. Of course you can design an ASIC for most stuff. And most performance-critical applications probably had ASICs designed for them by now, helping prove your point. SW and HW are not isolated either, and an algorithm optimized for old HW might be extremely inneficient on new HW, and vice-versa. And in any case, HW developments were in large part enabled by SW developments with logic synthesis, place and route, etc. HW development is SW development to a large extent today, though that was not your original point. What can't be argued against, however, is that both SW and HW improvements have made it much easier to create both HW and SW. Whether SW or HW has been most instrumental with this, I am not sure. They are tightly coupled: it's much easier to write a complex program with a modern compiler, but would you wait for it to compile on an old machine? Likewise for logic synthesis tools and HW simulators. Low-effort development can get you further, and that shows. I guess that's what you are complaining about. [1] https://en.wikipedia.org/wiki/Fast_Fourier_transform#Algorithms https://en.wikipedia.org/wiki/Fast_Fourier_transform#Algorit...
jhayward 6y ago
I think a better answer than I could give can be found at [1]. One pull quote they highlight: But the White House advisory report cited research, including a study of progress over a 15-year span on a benchmark production-planning task. Over that time, the speed of completing the calculations improved by a factor of 43 million. Of the total, a factor of roughly 1,000 was attributable to faster processor speeds, according to the research by Martin Grotschel, a German scientist and mathematician. Yet a factor of 43,000 was due to improvements in the efficiency of software algorithms. [1] https://cstheory.stackexchange.com/questions/12905/speedup-from-algorithmic-advances-vs-hardware https://cstheory.stackexchange.com/questions/12905/speedup-f...
- mpweiher 6y ago
  Fun example, thanks for pointing it out! I actually took Professor Grötschel's Linear Optimization course at TU Berlin, and the practical optimization task/competition we did for that course very much illustrates the point made in the answer to the stackexchange question you posted. Our team won the competition, beating the performance not just of the other student teams' programs, but also the program of the professor's assistants, by around an order of magnitude. How? By changing a single "<" (less than) to "<=" (less than or equal), which dramatically reduced the run-time of the dominant problem of the problem-set. This really miffed the professor quite a bit, because we were just a bunch of dumb CS majors taking a class in the far superior math department, but he was a good sport about it and we got a nice little prize in addition to our grade. It also helped that our program was still fastest without that one change, though now with only a tiny margin. The point being that, as the post notes, this is a single problem in a single, very specialized discipline, and this example absolutely does not generalize.
  - jhayward 6y ago
    I think that your original comment is the one that risks over-generalization. The "software gets slower" perception is in a very narrow technical niche: user interface/user experience for consumer applications. And it has a common cause: product managers and developers stuff as much in the the user experience as they can until performance suffers. But even within that over-stuffing phenomenon you can see individual components that have the same software improvement curve. In any area where computing performance has been critical - optimization, data management, simulation, physical modeling, statistical analysis, weather forecasting, genomics, computational medicine, imagery, etc. there have been many, many cases of software outpacing hardware in rate of improvement. Enough so that it is normal to expect it, and something to investigate for cause if it's not seen.