4 ms·

The SCIP quote the author tries to dispute is still very true. If you ask developers of a decently sized modern mature software system to list the main problems

The SCIP quote the author tries to dispute is still very true. If you ask developers of a decently sized modern mature software system to list the main problems they are encountering, they will probably name complexity, legacy code, lack of documentation, bugs, and not the performance. As to making readable programs run fast, this is the job for compilers. Compilers should perform more and more sophisticated code transformations for high-level languages with well-defined semantics, generating faster and faster code. The application-level programmers should not be burdened with details like CPU cache size.

tuatoru 5y ago
No. No. And maybe. What developers think does not matter if you want your software to be used. Responsiveness has always been the number one UI complaint. If your program is sluggish, that's virtually always because it's badly designed. That's your fault, not the compiler's. Application programmers don't need to know about CPU cache size because they should have designed their programs to perform on low-end hardware.
jcelerier 5y ago
> Compilers should perform more and more sophisticated code transformations Maybe but they don't, and my users asked for more performance yesterday (and the year before, and the year before that too, and likely the decade before that too). The problem with performance is that some people believe that it's a target that can be reached ; but the only acceptable target in most cases is "everything is instantaneous" - otherwise "power users" in your domain will complain and give you a bad reputation. But because people seemingly don't care or are in a position of power, I'm at a friend's home who asked me desperately to "look at why their computer doesn't work well, it's full of viruses" ; computer from 2012 takes 2min30 to go from "power button" to MS Word, there's literally nothing else installed than office and chrome and definitely no viruses, just windows services hogging all the resources because some people at Redmond likely believes in magic optimizing compilers.
- II2II 5y ago
  > and my users asked for more performance yesterday (and the year before, and the year before that too, and likely the decade before that too). That made me think of the traditional response of throwing bigger and faster computers at a problem. The thing is, much of the performance of those bigger and faster computers comes from architectural optimizations that can only be fully exploited if the program takes advantage of them. In some cases, putting the code through an optimizing compiler may be enough. In other cases, the program needs to be explicitly designed to take advantage of those optimizations. (The cache is an okay example of this. There was a time when caches did not exist. The introduction of caches would have sped up some programs more than others, depending upon the program's design.)
mlang23 5y ago
I am working in HPC, and I beg to differ. Yes, compilers should do optimisation whereever possible, but that is only a small percentage of the story. If the programmer doesnt know which algorithms to choose, the compiler is unable to fix that. If the programmer does not understand the underlying architecture at least a bit, the compiler is also likely going to fail fixing that. Take one of the most basic programming examples, fib. Depending on how your programming language might be implemented, the recursive definition is likely slower then necessary. So at the end of the lesson, you're being taught to make a loop out of the recursion, so save stack space and such. But usually, the lesson ends here. But this is sooooo far away from the best solution. Think 2x2 matrix and the power of n. No compiler will ever transform your naiv recursive fib to that. So no, the compiler is not the ultimate answer. If you dont manage to educate more programmers, we will continue to run slow software.
- wiz21c 5y ago
  yeah, but the more you optimize, the less your code is a way to communicate an idea on how/why (business wise) you did some things. I used to optimize some code. When I try to squeeze every CPU cycle, then I usually have to turn my algorithm upside down so that they "fit" correctly to the CPU architecture (cache, branch prediction, avoiding costly, instructions). In the end, algorithm cleverness, which is important to understand to read my code is totally muddied by optimization techniques here and there. Having the right balance between performance and readability is tough... Another example I cam across a while ago is matrix multiplication. See https://en.wikipedia.org/wiki/Computational_complexity_of_matrix_multiplication https://en.wikipedia.org/wiki/Computational_complexity_of_ma... The naive algorithm is super simple to understand and corresponds to the actual math definition. But the optimized algorithm, that's something totally different :-)
hsn915 5y ago
Deferring responsibility to other people is not a suitable attitude for a professional software engineer. There's a dangerous hidden assumption too in this mentality: the compiler is magic! It's not. The compiler mostly just does "micro optimizations" on the code. The most trivial example to demonstrate this is looping on a grid by column vs by row. The by-column iteration will be slower due to bad utilization of the CPU cache, and no compiler will ever rearrange the loop. Not because compilers are being purposely stupid. There's just no way to prove that the semantics will stay the same (except in the most trivial situation).
- rrss 5y ago
  > the most trivial example to demonstrate this is looping on a grid by column vs by row. The by-column iteration will be slower due to bad utilization of the CPU cache, and no compiler will ever rearrange the loop I think LLVM’s polyhedral optimization (Polly) probably will. I played with it a while back and found that if I handed it a naive matrix multiply with terrible locality, it was able to transform it to rearrange the iteration order and use tiling such that the naive w/ Polly ewas faster than any optimized version I could write. (my optimized versions probably weren’t great, but I tried the basics and got the normal speedups - Polly was just faster). https://polly.llvm.org/ https://polly.llvm.org/ https://releases.llvm.org/12.0.1/tools/polly/docs/UsingPollyWithClang.html https://releases.llvm.org/12.0.1/tools/polly/docs/UsingPolly...
  - hsn915 5y ago
    > matrix multiply probably one of the easy cases? I suppose the compiler must be able to somehow prove to itself that the code does not do anything else. What if, for example, within the loop you "printf(column)" or something like that? In this case re-arranging the loop would produce a different result, wouldn't it? So the compiler probably will not optimize it in that case, would it?
II2II 5y ago
Your point that the application level programmer should not be burdened with details like CPU cache size is something the article's author may agree with. They mentioned that cramming 17 floats into a cache that can hold 16 is (premature) optimization. On the other hand, the author is arguing that developers should take the presence of a CPU cache into consideration. The distinction is important. By failing to take the hardware into consideration it becomes much more difficult to solve performance issues down the road since an avenue to resolve it will likely involve rewriting disparate parts of code. Incidentally, this type of issue is unlikely to be addressed by optimizing compilers since it requires a knowledge of the runtime performance of disparate parts of code and may involve code that is not compiled at build time (e.g. the data structure may be passed to a third-party library).

tuatoru 5y ago

No. No. And maybe. What developers think does not matter if you want your software to be used. Responsiveness has always been the number one UI complaint. If your program is sluggish, that's virtually always because it's badly designed. That's your fault, not the compiler's. Application programmers don't need to know about CPU cache size because they should have designed their programs to perform on low-end hardware.

jcelerier 5y ago

> Compilers should perform more and more sophisticated code transformations Maybe but they don't, and my users asked for more performance yesterday (and the year before, and the year before that too, and likely the decade before that too). The problem with performance is that some people believe that it's a target that can be reached ; but the only acceptable target in most cases is "everything is instantaneous" - otherwise "power users" in your domain will complain and give you a bad reputation. But because people seemingly don't care or are in a position of power, I'm at a friend's home who asked me desperately to "look at why their computer doesn't work well, it's full of viruses" ; computer from 2012 takes 2min30 to go from "power button" to MS Word, there's literally nothing else installed than office and chrome and definitely no viruses, just windows services hogging all the resources because some people at Redmond likely believes in magic optimizing compilers.

II2II 5y ago
> and my users asked for more performance yesterday (and the year before, and the year before that too, and likely the decade before that too). That made me think of the traditional response of throwing bigger and faster computers at a problem. The thing is, much of the performance of those bigger and faster computers comes from architectural optimizations that can only be fully exploited if the program takes advantage of them. In some cases, putting the code through an optimizing compiler may be enough. In other cases, the program needs to be explicitly designed to take advantage of those optimizations. (The cache is an okay example of this. There was a time when caches did not exist. The introduction of caches would have sped up some programs more than others, depending upon the program's design.)

mlang23 5y ago

I am working in HPC, and I beg to differ. Yes, compilers should do optimisation whereever possible, but that is only a small percentage of the story. If the programmer doesnt know which algorithms to choose, the compiler is unable to fix that. If the programmer does not understand the underlying architecture at least a bit, the compiler is also likely going to fail fixing that. Take one of the most basic programming examples, fib. Depending on how your programming language might be implemented, the recursive definition is likely slower then necessary. So at the end of the lesson, you're being taught to make a loop out of the recursion, so save stack space and such. But usually, the lesson ends here. But this is sooooo far away from the best solution. Think 2x2 matrix and the power of n. No compiler will ever transform your naiv recursive fib to that. So no, the compiler is not the ultimate answer. If you dont manage to educate more programmers, we will continue to run slow software.

wiz21c 5y ago
yeah, but the more you optimize, the less your code is a way to communicate an idea on how/why (business wise) you did some things. I used to optimize some code. When I try to squeeze every CPU cycle, then I usually have to turn my algorithm upside down so that they "fit" correctly to the CPU architecture (cache, branch prediction, avoiding costly, instructions). In the end, algorithm cleverness, which is important to understand to read my code is totally muddied by optimization techniques here and there. Having the right balance between performance and readability is tough... Another example I cam across a while ago is matrix multiplication. See https://en.wikipedia.org/wiki/Computational_complexity_of_matrix_multiplication https://en.wikipedia.org/wiki/Computational_complexity_of_ma... The naive algorithm is super simple to understand and corresponds to the actual math definition. But the optimized algorithm, that's something totally different :-)

hsn915 5y ago

Deferring responsibility to other people is not a suitable attitude for a professional software engineer. There's a dangerous hidden assumption too in this mentality: the compiler is magic! It's not. The compiler mostly just does "micro optimizations" on the code. The most trivial example to demonstrate this is looping on a grid by column vs by row. The by-column iteration will be slower due to bad utilization of the CPU cache, and no compiler will ever rearrange the loop. Not because compilers are being purposely stupid. There's just no way to prove that the semantics will stay the same (except in the most trivial situation).

rrss 5y ago
> the most trivial example to demonstrate this is looping on a grid by column vs by row. The by-column iteration will be slower due to bad utilization of the CPU cache, and no compiler will ever rearrange the loop I think LLVM’s polyhedral optimization (Polly) probably will. I played with it a while back and found that if I handed it a naive matrix multiply with terrible locality, it was able to transform it to rearrange the iteration order and use tiling such that the naive w/ Polly ewas faster than any optimized version I could write. (my optimized versions probably weren’t great, but I tried the basics and got the normal speedups - Polly was just faster). https://polly.llvm.org/ https://polly.llvm.org/ https://releases.llvm.org/12.0.1/tools/polly/docs/UsingPollyWithClang.html https://releases.llvm.org/12.0.1/tools/polly/docs/UsingPolly...
- hsn915 5y ago
  > matrix multiply probably one of the easy cases? I suppose the compiler must be able to somehow prove to itself that the code does not do anything else. What if, for example, within the loop you "printf(column)" or something like that? In this case re-arranging the loop would produce a different result, wouldn't it? So the compiler probably will not optimize it in that case, would it?

II2II 5y ago

Your point that the application level programmer should not be burdened with details like CPU cache size is something the article's author may agree with. They mentioned that cramming 17 floats into a cache that can hold 16 is (premature) optimization. On the other hand, the author is arguing that developers should take the presence of a CPU cache into consideration. The distinction is important. By failing to take the hardware into consideration it becomes much more difficult to solve performance issues down the road since an avenue to resolve it will likely involve rewriting disparate parts of code. Incidentally, this type of issue is unlikely to be addressed by optimizing compilers since it requires a knowledge of the runtime performance of disparate parts of code and may involve code that is not compiled at build time (e.g. the data structure may be passed to a third-party library).