4 ms·

I always code with the mindset “the compiler is smarter than me.” No need to twist my logic around attempting to squeeze performance out of the processor - writ

I always code with the mindset “the compiler is smarter than me.” No need to twist my logic around attempting to squeeze performance out of the processor - write something understandable to humans, let the computer do what computers do.

qsort 11mo ago
> I always code with the mindset “the compiler is smarter than me.” Like with people in general, it depends on what compiler/interpreter we're talking about, I'll freely grant that clang is smarter than me, but CPython for sure isn't. :) More generally, canonicalization goes very far, but no farther than language semantics allows. Not even the notorious "sufficiently smart compiler" with infinite time can figure out what you don't tell it.
- manbitesdog 11mo ago
  To add to this, the low-level constraints also make this assumption noisy, no matter how smart the compiler is. On the CPython case, if you do `dis.dis('DAY = 24 * 60 * 60)` you will see that constant folding nicely converts it to `LOAD_CONST 86400`. However, if you try `dis.dis('ATOMS_IN_THE_WORLD = 10*50')` you will get LOAD_CONST 10, LOAD_CONST 50, BINARY_OP **.
  - zbentley 11mo ago
    > However, if you try `dis.dis('ATOMS_IN_THE_WORLD = 10*50')` you will get LOAD_CONST 10, LOAD_CONST 50... I do not get that, I get LOAD_CONST 500. Tested on: Python 3.9.3 MacOS (Apple provided), 3.13.3 (uv provided) MacOS and Linux, and 3.14.0 (uv provided) MacOS and Linux.
    - manbitesdog 10mo ago
      Sorry, I meant 10**50; HN formatting removed one asterisk
      - zbentley 10mo ago
        Ah, gotcha. Yep! Not constant-calculated at all!
adrianN 11mo ago
This is decent advice in general, but it pays off to try and express your logic in a way that is machine friendly. That mostly means thinking carefully about how you organize the data you work with. Optimizers generally don't change data structures or memory layout but that can make orders of magnitude difference in the performance of your program. It is also often difficult to refactor later.
- lou1306 11mo ago
  To make a more specific example, if you malloc()/free() within a loop, it's unlikely that the compiler will fix that for you. However, moving those calls outside of the loop (plus maybe add some realloc()s within, only if needed) is probably going to perform better.
  - adrianN 11mo ago
    That is something that can be easily found and usually fixed with trivial profiling. I'm more talking about data locality instead of pointer chasing. Once you set up a pointer-chasing data infrastructure changing that means rewriting most of your application.
- amiga386 11mo ago
  I find the same too. I find gcc and clang can inline functions, but can't decide to break apart a struct used only among those inlined functions and make every struct member a local variable, and then decide that one or more of those local variables should be allocated as a register for the full lifetime of the function, rather than spill onto the local stack. So if you use a messy solution where something that should be a struct and operated on with functions, is actually just a pile of local variables within a single function, and you use macros operating on local variables instead of inlineable functions operating on structs, you get massively better performance. e.g. /* slower */ struct foo { uint32_t a,b,c,d,e,f,g,h; } uint32_t do_thing(struct foo *foo) { return foo->a ^ foo->b ^ foo->c ^ foo->d; } void blah() { struct foo x; for (...) { x.e = do_thing(&x) ^ x.f; ... } } /* faster */ #define DO_THING (a^b^c^d) void blah() { uint32_t a,b,c,d,e,f,g,h; for (...) { e = DO_THING ^ f; ... } }
tonyhart7 11mo ago
also not all software need optimization to the bone pareto principle like always, dont need the best but good enough not every company is google level anyway
ErroneousBosh 11mo ago
You say that, but I was able to reduce the code size of some avr8 stuff I was working on by removing a whole bunch of instructions that zero out registers and then shift a value around. I don't it to literally shift the top byte 24 bits to the right and zero out the upper 24 bits, I just need it to pass the value in the top 8 bits direct to the next operation. I agree that most people are not writing hand-tuned avr8 assembly. Most people aren't attempting to do DSP on 8-bit AVRs either.
IshKebab 11mo ago
The fact that compilers are smart isn't an excuse to not think about performance at all. They can't change your program architecture, algorithms, memory access patterns, etc. You can mostly not think about super low level integer manipulation stuff though.
jaccola 11mo ago
I would take it one step further, often trying to eke out performance gains with clever tricks can hurt performance by causing you to "miss the forest for the trees". I work with Cuda kernels a lot for computer vision. I am able to consistently and significantly improve on the performance of research code without any fancy tricks, just with good software engineering practices. By organising variables into structs, improving naming, using helper functions, etc... the previously impenetrable code becomes so much clearer and the obvious optimisations reveal themselves. Not to say there aren't certain tricks / patterns / gotchas / low level hardware realities to keep in mind, of course.
flohofwoe 11mo ago
> I always code with the mindset “the compiler is smarter than me.” ...I don't know... for instance the MSVC compiler creates this output for the last two 'non-trivial' functions with '/Ox': add w8,w1,w0 cmp w0,#0 cseleq w0,w1,w8 Even beginner assembly coders on their first day wouldn't write such bullshit :) A better mindset is "don't trust the compiler for code that's actually performance sensitive". You shouldn't validate each line of compiler output, but at least for the 'hot areas' in the code base that definitely pays off, because sometimes compilers do really weird shit for no good reason (often because of 'interference' between unrelated optimizer passes) - and often you don't need to dig deep to stumble over weird output like in the example above.
- sumtechguy 11mo ago
  I see the msvc arm compiler has not improved much in 20 years. The msvc arm was pretty odd when we used it in ~2003. We did not trust it at all. Think we had to get 4 or so compiler fixes out of MS for that project plus 3 or 4 library fixes. The x86 one was pretty solid. We were targeting 4 different CPU platforms at the same time so we could find things like that decently quickly. Most of the the time it was something we did that was weird. But even then we would find them. That one looks like maybe the optimizer back filled a nop slot?
wavemode 11mo ago
This is very often true when your data is sitting right there on the stack. Though when your data is behind pointers, it's very easy to write code that the compiler can no longer figure out how to optimize.
wat10000 11mo ago
I would modify this a bit. Someone with decent computer architecture knowledge, tools, and time can generally do better than the compiler. But you generally won't, because you have a lot of other things to think about. So I'd state this as, "the compiler is more diligent and consistent than me." It's not so much that it can spot a for loop that's equivalent to a single add, but that it will spot it just about every time, so you don't have to worry about it.
stonemetal12 11mo ago
I go with "You are responsible for the algorithms, it is responsible for the code micro optimizations". The compiler can't optimize you out of an SQL N+1 situation, that is on me to avoid, but it is better than me at loop unrolling.
mamcx 11mo ago
> “the compiler is smarter than me.” This is true, but it also means "the compiler IS made for someone median smart, that now knows the machine". It works great for basic, simple, common code, and for code that is made with care for data structures. A total mess of code is another story. P.D: is similar to the query optimizers, that neither can outrun a terrible made schema and queries
moregrist 11mo ago
There are optimizations that a compiler can perform; usually these are code transformations. Modern optimizing compilers usually get these right. The optimizations that tend to have the most impact involve changes to the algorithm or data layout. Most compilers won’t do things like add a hash table to make a lookup O(1) or rearrange an array of structures to be a structure of arrays for better data locality. Coding with an eye for these optimizations is still a very good use of your time.
kragen 11mo ago
> I always code with the mindset “the compiler is smarter than me.” That mindset will last until the first or second time you walk through the compiler's assembly-language output. GCC and LLVM can find some amazing optimizations, but they also consistently miss very obvious optimization opportunities, often even on utterly trivial code. That isn't to say that it's worthwhile to hand-optimize all your code. But the compiler is very much not smarter than you, and if you do need to squeeze performance out of the processor, you can do a lot better than the compiler does. A good first step is to look at the stupid shit the compiler has spat out and massage your source code until the shit stinks less.

qsort 11mo ago

> I always code with the mindset “the compiler is smarter than me.” Like with people in general, it depends on what compiler/interpreter we're talking about, I'll freely grant that clang is smarter than me, but CPython for sure isn't. :) More generally, canonicalization goes very far, but no farther than language semantics allows. Not even the notorious "sufficiently smart compiler" with infinite time can figure out what you don't tell it.

manbitesdog 11mo ago
To add to this, the low-level constraints also make this assumption noisy, no matter how smart the compiler is. On the CPython case, if you do `dis.dis('DAY = 24 * 60 * 60)` you will see that constant folding nicely converts it to `LOAD_CONST 86400`. However, if you try `dis.dis('ATOMS_IN_THE_WORLD = 10*50')` you will get LOAD_CONST 10, LOAD_CONST 50, BINARY_OP **.
- zbentley 11mo ago
  > However, if you try `dis.dis('ATOMS_IN_THE_WORLD = 10*50')` you will get LOAD_CONST 10, LOAD_CONST 50... I do not get that, I get LOAD_CONST 500. Tested on: Python 3.9.3 MacOS (Apple provided), 3.13.3 (uv provided) MacOS and Linux, and 3.14.0 (uv provided) MacOS and Linux.
  - manbitesdog 10mo ago
    Sorry, I meant 10**50; HN formatting removed one asterisk
    - zbentley 10mo ago
      Ah, gotcha. Yep! Not constant-calculated at all!

adrianN 11mo ago

This is decent advice in general, but it pays off to try and express your logic in a way that is machine friendly. That mostly means thinking carefully about how you organize the data you work with. Optimizers generally don't change data structures or memory layout but that can make orders of magnitude difference in the performance of your program. It is also often difficult to refactor later.

lou1306 11mo ago
To make a more specific example, if you malloc()/free() within a loop, it's unlikely that the compiler will fix that for you. However, moving those calls outside of the loop (plus maybe add some realloc()s within, only if needed) is probably going to perform better.
- adrianN 11mo ago
  That is something that can be easily found and usually fixed with trivial profiling. I'm more talking about data locality instead of pointer chasing. Once you set up a pointer-chasing data infrastructure changing that means rewriting most of your application.
amiga386 11mo ago
I find the same too. I find gcc and clang can inline functions, but can't decide to break apart a struct used only among those inlined functions and make every struct member a local variable, and then decide that one or more of those local variables should be allocated as a register for the full lifetime of the function, rather than spill onto the local stack. So if you use a messy solution where something that should be a struct and operated on with functions, is actually just a pile of local variables within a single function, and you use macros operating on local variables instead of inlineable functions operating on structs, you get massively better performance. e.g. /* slower */ struct foo { uint32_t a,b,c,d,e,f,g,h; } uint32_t do_thing(struct foo *foo) { return foo->a ^ foo->b ^ foo->c ^ foo->d; } void blah() { struct foo x; for (...) { x.e = do_thing(&x) ^ x.f; ... } } /* faster */ #define DO_THING (a^b^c^d) void blah() { uint32_t a,b,c,d,e,f,g,h; for (...) { e = DO_THING ^ f; ... } }

tonyhart7 11mo ago

also not all software need optimization to the bone pareto principle like always, dont need the best but good enough not every company is google level anyway

ErroneousBosh 11mo ago

You say that, but I was able to reduce the code size of some avr8 stuff I was working on by removing a whole bunch of instructions that zero out registers and then shift a value around. I don't it to literally shift the top byte 24 bits to the right and zero out the upper 24 bits, I just need it to pass the value in the top 8 bits direct to the next operation. I agree that most people are not writing hand-tuned avr8 assembly. Most people aren't attempting to do DSP on 8-bit AVRs either.

IshKebab 11mo ago

The fact that compilers are smart isn't an excuse to not think about performance at all. They can't change your program architecture, algorithms, memory access patterns, etc. You can mostly not think about super low level integer manipulation stuff though.

jaccola 11mo ago

I would take it one step further, often trying to eke out performance gains with clever tricks can hurt performance by causing you to "miss the forest for the trees". I work with Cuda kernels a lot for computer vision. I am able to consistently and significantly improve on the performance of research code without any fancy tricks, just with good software engineering practices. By organising variables into structs, improving naming, using helper functions, etc... the previously impenetrable code becomes so much clearer and the obvious optimisations reveal themselves. Not to say there aren't certain tricks / patterns / gotchas / low level hardware realities to keep in mind, of course.

flohofwoe 11mo ago

> I always code with the mindset “the compiler is smarter than me.” ...I don't know... for instance the MSVC compiler creates this output for the last two 'non-trivial' functions with '/Ox': add w8,w1,w0 cmp w0,#0 cseleq w0,w1,w8 Even beginner assembly coders on their first day wouldn't write such bullshit :) A better mindset is "don't trust the compiler for code that's actually performance sensitive". You shouldn't validate each line of compiler output, but at least for the 'hot areas' in the code base that definitely pays off, because sometimes compilers do really weird shit for no good reason (often because of 'interference' between unrelated optimizer passes) - and often you don't need to dig deep to stumble over weird output like in the example above.

sumtechguy 11mo ago
I see the msvc arm compiler has not improved much in 20 years. The msvc arm was pretty odd when we used it in ~2003. We did not trust it at all. Think we had to get 4 or so compiler fixes out of MS for that project plus 3 or 4 library fixes. The x86 one was pretty solid. We were targeting 4 different CPU platforms at the same time so we could find things like that decently quickly. Most of the the time it was something we did that was weird. But even then we would find them. That one looks like maybe the optimizer back filled a nop slot?

wavemode 11mo ago

This is very often true when your data is sitting right there on the stack. Though when your data is behind pointers, it's very easy to write code that the compiler can no longer figure out how to optimize.

wat10000 11mo ago

I would modify this a bit. Someone with decent computer architecture knowledge, tools, and time can generally do better than the compiler. But you generally won't, because you have a lot of other things to think about. So I'd state this as, "the compiler is more diligent and consistent than me." It's not so much that it can spot a for loop that's equivalent to a single add, but that it will spot it just about every time, so you don't have to worry about it.

stonemetal12 11mo ago

I go with "You are responsible for the algorithms, it is responsible for the code micro optimizations". The compiler can't optimize you out of an SQL N+1 situation, that is on me to avoid, but it is better than me at loop unrolling.

mamcx 11mo ago

> “the compiler is smarter than me.” This is true, but it also means "the compiler IS made for someone median smart, that now knows the machine". It works great for basic, simple, common code, and for code that is made with care for data structures. A total mess of code is another story. P.D: is similar to the query optimizers, that neither can outrun a terrible made schema and queries

moregrist 11mo ago

There are optimizations that a compiler can perform; usually these are code transformations. Modern optimizing compilers usually get these right. The optimizations that tend to have the most impact involve changes to the algorithm or data layout. Most compilers won’t do things like add a hash table to make a lookup O(1) or rearrange an array of structures to be a structure of arrays for better data locality. Coding with an eye for these optimizations is still a very good use of your time.

kragen 11mo ago

> I always code with the mindset “the compiler is smarter than me.” That mindset will last until the first or second time you walk through the compiler's assembly-language output. GCC and LLVM can find some amazing optimizations, but they also consistently miss very obvious optimization opportunities, often even on utterly trivial code. That isn't to say that it's worthwhile to hand-optimize all your code. But the compiler is very much not smarter than you, and if you do need to squeeze performance out of the processor, you can do a lot better than the compiler does. A good first step is to look at the stupid shit the compiler has spat out and massage your source code until the shit stinks less.