4 ms·

> asking for forgiveness (dereferencing a null pointer and then recovering) instead of permission (checking if the pointer is null before dereferencing it) is a

> asking for forgiveness (dereferencing a null pointer and then recovering) instead of permission (checking if the pointer is null before dereferencing it) is an optimization. Comparing all pointers with null would slow down execution when the pointer isn’t null, i.e. in the majority of cases. In contrast, signal handling is zero-cost until the signal is generated, which happens exceedingly rarely in well-written programs. Is this actually a real optimization? I understand the principal, that you can bypass explicit checks by using exception handlers and then massage the stack/registers back to a running state, but does this actually optimize speed? A null pointer check is literally a single TEST on a register, followed by a conditional jump the branch predictor is 99.9% of the time going to know what to do with. How much processing time is using an exception actually going to save? Or is there a better example?

oguz-ismail 2y ago
Signal handling is done anyway. The cost of null pointer checks is a net overhead, if minuscule.
liontwist 2y ago
> A null pointer check is literally a single TEST on a register On every pointer deref in your entire program. Not for release mode.
gwbas1c 2y ago
> Is this actually a real optimization? No... And yes. No: Because throwing and catching the null pointer exception is hideously slow compared to doing a null check. In Java / C#, the exception is an allocated object, and the stack is walked to generate a stack trace. This is in addition to any additional lower-level overhead (panic) that I don't understand the details well enough to explain. Yes: If, in practice, the pointer is never null, (and thus a null pointer is truly an exceptional situation,) carefully-placed exception handlers are an optimization. Although, yes, the code will technically be faster because it's not doing null checks, the most important optimization is developer time and code cleanliness. The developer doesn't waste time adding redundant null checks, and the next developer finds code that is easier to read because it isn't littered with redundant null checks.
- mr_00ff00 2y ago
  “Most important optimization is developer time and code cleaniness” True for 99% of programming jobs, but if you are worried about the speed of null checks, you are in that 1%. In high frequency trading, if you aren’t first your last and this is the exact type of code optimizations you need for the “happy path”
- wging 2y ago
  If you're actually paying a significant cost generating stack traces for NPEs, there's a JVM option to deal with that (-XX:-OmitStackTraceInFastThrow). It still generates a stack trace the first time; if you're able to go search for that first one it shouldn't be a problem for debugging.
- LorenPechtel 2y ago
  Yup, three is definitely value to the code not doing things it doesn't need to. I also find it clearer if the should-never-happen null check is in the form of an assertion. You know anything being tested in an Assert is a should never happen path, you don't need to consider why it's being done.
  - gwbas1c 2y ago
    > You know anything being tested in an Assert is a should never happen path, you don't need to consider why it's being done That's also rather... Redundant in modern null-safe (or similar) languages. IE, Swift and C# have compiler null checking where the method can indicate if an argument accepts null. There's no point in assertions; thus the null reference exceptions do their job well here. Rust's option type (which does almost the same thing but trolls love to argue semantics) also is a situation where the compiler makes it hard to pass the "null equivalent." (I'm not sure if a creative programmer can trick the runtime into passing a null pointer where it's unexpected, but I don't understand unsafe Rust well enough to judge that.) But if you need to defend against that, I think Panics can do it.
    - LorenPechtel 2y ago
      You're assuming that you can always make your declarations correctly express whether something can be null. Let's consider a real world example I hit: Every cell has a concept of it's neighbors. Now, the world isn't infinite, some of those pointers are null. But there's a boundary layer of edge-of-the-world cells to keep you from going too far. Thus, in the real world null can never occur--but there are a gazillion places where it would have to test for a null that should never happen. Or simply Assert in the routine itself.
nickff 2y ago
The OP is offering terrible advice based on a falsehood they believe about null pointers. In many applications (including the STM32H743 microcontroller that I am currently working on), address zero (which is how "NULL" is defined by default in my IDE) points to RAM or FLASH. In my current application, NULL is ITCM (instruction tightly coupled memory), and it's where I've put my interrupt vector table. If I read it, I don't get an error, but I may get dangerously wrong data.
- pfyra 2y ago
  I work with the same mcu. You can set up the MPU to catch null pointer dererences so they don't pass silently.
- russdill 2y ago
  Not only that, if you're referencing an element in a very large structure or array, the base address may be zero, but the actual access may be several pages past that.
- LorenPechtel 2y ago
  I disagree. You're looking at embedded code which very well might not be running with memory segmentation. If you have no hardware safety you must check your pointers, period. But few of us are in that situation. Personally, I haven't touched an environment without hardware safety in 20 years.
Aloisius 2y ago
OpenJVM does it, iirc. If the handler is triggered too often at a location, it will swap back to emitting null checks though since it is rather expensive. Of course, there's a big difference between doing it in a VM and doing it in a random piece of software.
toast0 2y ago
Sure, the cost of the check is small, and if you actually hit a null pointer, the cost is much higher if it's flagged by the MMU instead of a simple check. But you're saving probably two bytes in the instruction stream for the test and conditional jump (more if you don't have variable length instructions), and maybe that adds up over your whole program so you can keep meaningfully more code in cache.
- LorenPechtel 2y ago
  More important is the branch predictor. Sometimes you take the hit of a failed predict and you also stuck another entry in the predictor table, making it more likely some other code will suffer a failed branch predict.
bjackman 2y ago
Regardless of whether it's faster, it's an extremely bad idea in a C program, for many of the reasons the author outlines. It's UB so you are forever gonna be worrying about what weird behaviour the compiler might create. Classic example: compiler infers some case where the pointer is always NULL and then just deletes all the code that would run in that case. Plus, now you have to do extremely sketchy signal handling.
Validark 2y ago
I would be very interested to see optimizations based on this. Could the compiler emit faster code with this idea? What impact does this have on auto-vectorization like using `vgather`?
Gibbon1 2y ago
My takes are Unlike crummy barely more than a macro assembler compilers of yore. A modern compiler will optimize away a lot of null pointer checks. Superscalar processors can chew gum and walk at the same time while juggling and hula hooping. That is if they also don't decide the null check isn't meaningful and toss it away. And yeah checking for null is a trivial operation they'll do while busy with other things. And the performance of random glue code running on a single core isn't where any of the wins are when it comes to speed and hasn't been for 15 years.
LorenPechtel 2y ago
Conditional jumps tend to carry performance penalties because they tend to trash the look-ahead. And are pretty much forbidden in some cryptographic code where you want constant time execution no matter what. Also, so long as the hardware ensures that you don't get away with using the null I would think that not doing the test is the proper approach. Hitting a null is a bug, period (although it could be a third party at fault.) And I would say that adding a millisecond to the bugged path (but, library makers, please always make safe routines available! More than once I've seen stuff that provides no test for whether something exists. I really didn't like it the day I found the routine that returned the index of the supplied value or -1 if it didn't exist--but it wasn't exposed, only the return the index or throw if it didn't exist was accessible.) vs saving a nanosecond on the correct path is the right decision.
- UncleMeat 2y ago
  In languages like java, jits are really good at optimizing the happy path and outlining these rarely taken cases. In a language like c++ the branch predictor has absolutely no trouble blasting right through branches that are almost always taken in one direction with minimal performance waste.

oguz-ismail 2y ago

Signal handling is done anyway. The cost of null pointer checks is a net overhead, if minuscule.

liontwist 2y ago

> A null pointer check is literally a single TEST on a register On every pointer deref in your entire program. Not for release mode.

gwbas1c 2y ago

> Is this actually a real optimization? No... And yes. No: Because throwing and catching the null pointer exception is hideously slow compared to doing a null check. In Java / C#, the exception is an allocated object, and the stack is walked to generate a stack trace. This is in addition to any additional lower-level overhead (panic) that I don't understand the details well enough to explain. Yes: If, in practice, the pointer is never null, (and thus a null pointer is truly an exceptional situation,) carefully-placed exception handlers are an optimization. Although, yes, the code will technically be faster because it's not doing null checks, the most important optimization is developer time and code cleanliness. The developer doesn't waste time adding redundant null checks, and the next developer finds code that is easier to read because it isn't littered with redundant null checks.

mr_00ff00 2y ago
“Most important optimization is developer time and code cleaniness” True for 99% of programming jobs, but if you are worried about the speed of null checks, you are in that 1%. In high frequency trading, if you aren’t first your last and this is the exact type of code optimizations you need for the “happy path”
wging 2y ago
If you're actually paying a significant cost generating stack traces for NPEs, there's a JVM option to deal with that (-XX:-OmitStackTraceInFastThrow). It still generates a stack trace the first time; if you're able to go search for that first one it shouldn't be a problem for debugging.
LorenPechtel 2y ago
Yup, three is definitely value to the code not doing things it doesn't need to. I also find it clearer if the should-never-happen null check is in the form of an assertion. You know anything being tested in an Assert is a should never happen path, you don't need to consider why it's being done.
- gwbas1c 2y ago
  > You know anything being tested in an Assert is a should never happen path, you don't need to consider why it's being done That's also rather... Redundant in modern null-safe (or similar) languages. IE, Swift and C# have compiler null checking where the method can indicate if an argument accepts null. There's no point in assertions; thus the null reference exceptions do their job well here. Rust's option type (which does almost the same thing but trolls love to argue semantics) also is a situation where the compiler makes it hard to pass the "null equivalent." (I'm not sure if a creative programmer can trick the runtime into passing a null pointer where it's unexpected, but I don't understand unsafe Rust well enough to judge that.) But if you need to defend against that, I think Panics can do it.
  - LorenPechtel 2y ago
    You're assuming that you can always make your declarations correctly express whether something can be null. Let's consider a real world example I hit: Every cell has a concept of it's neighbors. Now, the world isn't infinite, some of those pointers are null. But there's a boundary layer of edge-of-the-world cells to keep you from going too far. Thus, in the real world null can never occur--but there are a gazillion places where it would have to test for a null that should never happen. Or simply Assert in the routine itself.

nickff 2y ago

The OP is offering terrible advice based on a falsehood they believe about null pointers. In many applications (including the STM32H743 microcontroller that I am currently working on), address zero (which is how "NULL" is defined by default in my IDE) points to RAM or FLASH. In my current application, NULL is ITCM (instruction tightly coupled memory), and it's where I've put my interrupt vector table. If I read it, I don't get an error, but I may get dangerously wrong data.

pfyra 2y ago
I work with the same mcu. You can set up the MPU to catch null pointer dererences so they don't pass silently.
russdill 2y ago
Not only that, if you're referencing an element in a very large structure or array, the base address may be zero, but the actual access may be several pages past that.
LorenPechtel 2y ago
I disagree. You're looking at embedded code which very well might not be running with memory segmentation. If you have no hardware safety you must check your pointers, period. But few of us are in that situation. Personally, I haven't touched an environment without hardware safety in 20 years.

Aloisius 2y ago

OpenJVM does it, iirc. If the handler is triggered too often at a location, it will swap back to emitting null checks though since it is rather expensive. Of course, there's a big difference between doing it in a VM and doing it in a random piece of software.

toast0 2y ago

Sure, the cost of the check is small, and if you actually hit a null pointer, the cost is much higher if it's flagged by the MMU instead of a simple check. But you're saving probably two bytes in the instruction stream for the test and conditional jump (more if you don't have variable length instructions), and maybe that adds up over your whole program so you can keep meaningfully more code in cache.

LorenPechtel 2y ago
More important is the branch predictor. Sometimes you take the hit of a failed predict and you also stuck another entry in the predictor table, making it more likely some other code will suffer a failed branch predict.

bjackman 2y ago

Regardless of whether it's faster, it's an extremely bad idea in a C program, for many of the reasons the author outlines. It's UB so you are forever gonna be worrying about what weird behaviour the compiler might create. Classic example: compiler infers some case where the pointer is always NULL and then just deletes all the code that would run in that case. Plus, now you have to do extremely sketchy signal handling.

Validark 2y ago

I would be very interested to see optimizations based on this. Could the compiler emit faster code with this idea? What impact does this have on auto-vectorization like using `vgather`?

Gibbon1 2y ago

My takes are Unlike crummy barely more than a macro assembler compilers of yore. A modern compiler will optimize away a lot of null pointer checks. Superscalar processors can chew gum and walk at the same time while juggling and hula hooping. That is if they also don't decide the null check isn't meaningful and toss it away. And yeah checking for null is a trivial operation they'll do while busy with other things. And the performance of random glue code running on a single core isn't where any of the wins are when it comes to speed and hasn't been for 15 years.

LorenPechtel 2y ago

Conditional jumps tend to carry performance penalties because they tend to trash the look-ahead. And are pretty much forbidden in some cryptographic code where you want constant time execution no matter what. Also, so long as the hardware ensures that you don't get away with using the null I would think that not doing the test is the proper approach. Hitting a null is a bug, period (although it could be a third party at fault.) And I would say that adding a millisecond to the bugged path (but, library makers, please always make safe routines available! More than once I've seen stuff that provides no test for whether something exists. I really didn't like it the day I found the routine that returned the index of the supplied value or -1 if it didn't exist--but it wasn't exposed, only the return the index or throw if it didn't exist was accessible.) vs saving a nanosecond on the correct path is the right decision.

UncleMeat 2y ago
In languages like java, jits are really good at optimizing the happy path and outlining these rarely taken cases. In a language like c++ the branch predictor has absolutely no trouble blasting right through branches that are almost always taken in one direction with minimal performance waste.