4 ms·

Yes. Malloc is a really good example of this, where malloc returning NULL indicates a fundamental resource is not available. If you dereference the pointer retu

Yes. Malloc is a really good example of this, where malloc returning NULL indicates a fundamental resource is not available. If you dereference the pointer returned by malloc under those conditions, your program crashes, just like if you were to not catch an exception thrown at instantiation-time with a reference type. You should absolutely use NULL to represent the absence of some fundamental resource, like memory or a device. To the extent that a device is memory-mapped, it's the same thing. NULL is just a tool. It's just a way for you to express some property of your API. That said, some of the strategies outlined in the article could be very useful for avoiding use of pointers. Use of a class type implementing an interface as a "null type" is a very good strategy. Such a type could guarantee successful instantiation, and so avoid the need to throw exceptions or otherwise crash entirely. You could architect an application with those strategies and completely avoid the use of NULL and nullptr entirely. But where your program is dependent on a finite resource, you must have a way to represent the absence of that resource. NULL and nullptr are convenient ways to do that without a ton of extra code. So no, don't avoid NULL.

aidenn0 11y ago
Dereferrencing NULL will not always cause your program to crash.
- mikekchar 11y ago
  You have intrigued me, as what you say is true, but I wish you would elaborate. There are some architectures for which dereferencing an address of 0 does not cause an exception. If you are writing code for those architectures (or code that is meant to be portable), then you can't rely on dereferencing NULL to crash. This is universally well known amongst programmers who work on those architectures. It's been more than a decade since I did any C/C++, so I wonder if you were referring so something else that I'm not aware of.
  - saurik 11y ago
    Generally it is not an "architecture" that makes this fail, it is a property of the operating system or even only the linker. If you look at a Mach-O executable on OS X, you will see a __PAGEZERO segment that explicitly causes the dynamic linker to put an invalid block of memory at address 0: for backwards compatibility reasons, in 32-bit processes, without that totally optional load command, and even with that command if you choose to override it at runtime using mmap or any number of mach memory management functions, you can allocate memory at that address. (Apple mitigates this in 64-bit processes by having the operating system disallow mapping memory to this location.) In fact, this is a technique commonly used in many kernel exploits on both Linux and OS X, and was even used just a few months ago in the "tpwn" exploit of OS X 10.10.5, prompting the researcher who disclosed that vulnerability to also release NULLGuard, whose entire purpose is to help protect your system by disallowing processes from executing if they are missing a __PAGEZERO segment. https://github.com/kpwn/NULLGuard https://github.com/kpwn/NULLGuard
    - dllthomas 11y ago
      It can also be the compiler. If it observes you dereferencing a pointer, it can elsewhere assume that pointer is not null. If that's not the case, things can get messy.
- MichaelBurge 11y ago
  I agree it's possible to design a compiler that doesn't crash on a NULL pointer and still call the language C. Or certain optimizations could trigger that confuse a naive C programmer. But modern hardware is frequently designed so that dereferencing a zero pointer causes a crash. Even embedded systems will often treat the first couple hundred bytes or so after 0 specially, since it catches so many bugs. I think it's generally safe to assume you'll get a crash if you actually dereference a null pointer.
  - saurik 11y ago
    This is a false assumption that directly leads to exploitable software. See my reply to a sibling comment for more details, but I will add here (as you went in a slightly different direction) that on many embedded systems (such as ARM) the address 0 is not "special" in that it is disallowed but is "special" in that it stores particularly important control structures for the CPU, such as its interrupt vector table: one of the bootrom exploits used on the iPhone took advantage of this (though sadly I'm not remembering exactly which one, though the documentation for 24kpwn mentions this mapping). It isn't until you are running in a high-level operating system, and even sometimes not even then (such as current versions of OS X, which rely on cooperation between the dynamic linker and the compiler to implement this feature, as I describe in my linked reply), that you have some (limited...) safety from NULL pointer dereferences. (I say "limited", as sometimes people store egregiously large objects that are larger than a couple pages of memory and some dereferences are negative.) https://news.ycombinator.com/item?id=10531319 https://news.ycombinator.com/item?id=10531319 https://www.theiphonewiki.com/wiki/0x24000_Segment_Overflow https://www.theiphonewiki.com/wiki/0x24000_Segment_Overflow
    - aidenn0 11y ago
      Funny story: a customer porting from an OS that didn't leave NULL unmapped to an OS that did, asked us to tell them how to map the zero page, as their program had spurious NULL pointer accesses that they didn't want to fix.
derefr 11y ago
NULL is just a tool. Non-optional nullability on all pointer types is a blight. 99% of APIs want to say "returns a guaranteed-valid pointer", and "takes a guaranteed-valid pointer", and have the language enforce those guarantees. C and its siblings just give you a "might-be-valid pointer" type, which introduces NULL into every API you create, whether you want it there or not. I do agree that there's no fundamental difference between using a "might-be-valid pointer" vs. a sum-type (Option/Maybe/etc.) to indicate nullability. However, it's usually easier to get static analysis of the sum type—most type-inferencing compilers can ensure that when you destructure a sum type, you provide match-clauses for every possible instantiation of the type (i.e., for an Option type, they ensure you do the equivalent of a NULL check, and handle both the NULL and non-NULL cases), and so forth. What I'd really like to see, though, is a language with data-parameterized "types" that unfold (as a last resort, when the concrete type cannot be inferred at compile-time) into runtime data-comparison branches. You should be able to have a raw may-be-valid address with no metadata tag, and then treat that data as having a separate type (a Just or a Nothing) depending on whether the memory address itself is nonzero. You write match clauses; the compiler turns those into NULL checks. (As an aside, such a language would also allow for things like a type for "Float that only has an integer component", or "Even number", or "Positive number", or "Number in the range 5..23", or "String of length 5", or "String that encodes a valid Unicode codepoint sequence", etc. The type system would become your swiss-army knife for expressing invariants.)
- johncolanduoni 11y ago
  > What I'd really like to see, though, is a language with parameterized typing with hybrid static+runtime enforcement. You should be able to be holding a raw address in memory, and then have that become one of two strong types at the language level depending on the whether that memory address is nonzero—"lifting" the NULL check into the type system, rather than replacing the NULL check with extra type annotation. In Rust, the compiler optimizes pointer types wrapped in Options to nullable pointers. Is this the kind of thing you meant?
  - derefr 11y ago
    Possibly—does the Rust FFI allow you to import a C library, writing its foreign-function declarations such that the nullable pointers it returns are transformed into, from Rust's perspective, Option-typed non-nullable pointers, with no runtime memory overhead? And then, if so, does Rust further guarantee that those "virtual" Option-typed pointers can be passed back to the same API "just the way it sent them"? That's sort of what I mean, here: the nullable ptr as the "canonical, machine-level" representation, with the type serving as a language-level abstraction, effectively a "lie" because it has no real effect on the type system, and only serves as syntax sugar for generating runtime checks.
johncolanduoni 11y ago
The problem is NULL does not "fail fast". You could get a NULL pointer as an argument, and then not use that value until much later. Now when the bug shows up, you'll have to work back through your code to even figure out where it came from. > Use of a class type implementing an interface as a "null type" is a very good strategy. Have you ever programmed Objective-C? It uses the null object pattern (called "nil") for all Objective-C objects. This makes it really easy to put big logic errors into your code, since calling upon a nil will just give you more nil. So now you have to wait even longer until a fatal error pops up, and then find where on earth the original nil came from. > NULL and nullptr are convenient ways to do that without a ton of extra code. How do you figure? If malloc can return NULL and you want a sane error message, you need to write a NULL check whenever you do a malloc. Otherwise, your user will just get a segfault, instead of a notification that it was memory they ran out of. So why not do that by default, and for the few places where you have prepared yourself to handle out of memory conditions use a different malloc which returns a value that must be explicitly checked?
- jschwartzi 11y ago
  Because I don't have a system library that provides a different malloc. It's a property of my API that malloc returns NULL and I don't have any control over that. What I do have control over is what I do with data I get out of malloc. I have to check, somehow, for abnormal conditions. That is a requirement for using the malloc API. I get that this makes it hard to debug in some cases, so I might avoid that by religiously checking for NULL return from malloc and crashing immediately with a sensible error message. In the cases where getting a NULL return is not an error, like if I wrote a program to use as much heap as it can, I would ignore the NULL and proceed with the resources I can actually get.
  - johncolanduoni 11y ago
    What stops you from writing an API on top of the system malloc that has two malloc functions like I described above, with return types that have unambiguous nullity? Then your religious NULL checks won't be duplicated (and you won't be able to stray from the righteous path and forget any).
mikeash 11y ago
An optional type's "none" value represents the same properties that NULL does. The difference is that the optional type better expresses what's going on. NULL breaks the type system. Now every pointer type is actually two types: the normal type, and the NULL type. You can dereference or call methods on one, but not the other, and only a runtime check will tell you what you're dealing with. Any time you see: Object * You cannot read that as "pointer to Object." You have to read it as "pointer to Object, or NULL." This causes three big problems. One is that it's way too easy to forget to check for NULL. The second, related problem is that without a way to express a non-nullable pointer type, you have to rely on documentation and convention to express the difference. The third is that since only pointer types are nullable, you have to reinvent the wheel any time you you want to express the concept of "null" for a non-pointer type. To illustrate, you point out malloc which returns NULL when it runs out of address space. What's the appropriate return type? void * Now let's say you had some other memory allocation call which could never return failure to the caller. Maybe it aborts execution on failure, or maybe it retries continually until memory becomes available. What's the appropriate return type? void * Oops. Same problem with parameters. Here's a function: void f(void *ptr) Are you allowed to pass NULL? Who knows, go RTFM. If the answer is "no," what happens if you pass NULL anyway? The compiler certainly won't catch it, and who knows what will happen. NULL is "just a tool" but it's not a very good tool. It breaks the type system in a fundamental and painful way. Edit: whew, HN's markup makes it kind of hard to talk about C pointers!
- jschwartzi 11y ago
  Right, but in malloc's case, it needs to express something about the fundamental type of all software, which is memory. There is nothing more fundamental to the software system. When you run out of memory and fail to check for that condition, the only right and proper thing to do is crash. NULL is a handy way to enforce that fundamentally, in that dereferencing a null pointer guarantees a crash. Part of programming is about memory management. You can hide it with a sophisticated compiler but it's always going to be there, and there are always going to be two types of memory: that which is available, and that which is not.
  - 11y ago

aidenn0 11y ago

Dereferrencing NULL will not always cause your program to crash.

mikekchar 11y ago
You have intrigued me, as what you say is true, but I wish you would elaborate. There are some architectures for which dereferencing an address of 0 does not cause an exception. If you are writing code for those architectures (or code that is meant to be portable), then you can't rely on dereferencing NULL to crash. This is universally well known amongst programmers who work on those architectures. It's been more than a decade since I did any C/C++, so I wonder if you were referring so something else that I'm not aware of.
- saurik 11y ago
  Generally it is not an "architecture" that makes this fail, it is a property of the operating system or even only the linker. If you look at a Mach-O executable on OS X, you will see a __PAGEZERO segment that explicitly causes the dynamic linker to put an invalid block of memory at address 0: for backwards compatibility reasons, in 32-bit processes, without that totally optional load command, and even with that command if you choose to override it at runtime using mmap or any number of mach memory management functions, you can allocate memory at that address. (Apple mitigates this in 64-bit processes by having the operating system disallow mapping memory to this location.) In fact, this is a technique commonly used in many kernel exploits on both Linux and OS X, and was even used just a few months ago in the "tpwn" exploit of OS X 10.10.5, prompting the researcher who disclosed that vulnerability to also release NULLGuard, whose entire purpose is to help protect your system by disallowing processes from executing if they are missing a __PAGEZERO segment. https://github.com/kpwn/NULLGuard https://github.com/kpwn/NULLGuard
  - dllthomas 11y ago
    It can also be the compiler. If it observes you dereferencing a pointer, it can elsewhere assume that pointer is not null. If that's not the case, things can get messy.
MichaelBurge 11y ago
I agree it's possible to design a compiler that doesn't crash on a NULL pointer and still call the language C. Or certain optimizations could trigger that confuse a naive C programmer. But modern hardware is frequently designed so that dereferencing a zero pointer causes a crash. Even embedded systems will often treat the first couple hundred bytes or so after 0 specially, since it catches so many bugs. I think it's generally safe to assume you'll get a crash if you actually dereference a null pointer.
- saurik 11y ago
  This is a false assumption that directly leads to exploitable software. See my reply to a sibling comment for more details, but I will add here (as you went in a slightly different direction) that on many embedded systems (such as ARM) the address 0 is not "special" in that it is disallowed but is "special" in that it stores particularly important control structures for the CPU, such as its interrupt vector table: one of the bootrom exploits used on the iPhone took advantage of this (though sadly I'm not remembering exactly which one, though the documentation for 24kpwn mentions this mapping). It isn't until you are running in a high-level operating system, and even sometimes not even then (such as current versions of OS X, which rely on cooperation between the dynamic linker and the compiler to implement this feature, as I describe in my linked reply), that you have some (limited...) safety from NULL pointer dereferences. (I say "limited", as sometimes people store egregiously large objects that are larger than a couple pages of memory and some dereferences are negative.) https://news.ycombinator.com/item?id=10531319 https://news.ycombinator.com/item?id=10531319 https://www.theiphonewiki.com/wiki/0x24000_Segment_Overflow https://www.theiphonewiki.com/wiki/0x24000_Segment_Overflow
  - aidenn0 11y ago
    Funny story: a customer porting from an OS that didn't leave NULL unmapped to an OS that did, asked us to tell them how to map the zero page, as their program had spurious NULL pointer accesses that they didn't want to fix.

derefr 11y ago

NULL is just a tool. Non-optional nullability on all pointer types is a blight. 99% of APIs want to say "returns a guaranteed-valid pointer", and "takes a guaranteed-valid pointer", and have the language enforce those guarantees. C and its siblings just give you a "might-be-valid pointer" type, which introduces NULL into every API you create, whether you want it there or not. I do agree that there's no fundamental difference between using a "might-be-valid pointer" vs. a sum-type (Option/Maybe/etc.) to indicate nullability. However, it's usually easier to get static analysis of the sum type—most type-inferencing compilers can ensure that when you destructure a sum type, you provide match-clauses for every possible instantiation of the type (i.e., for an Option type, they ensure you do the equivalent of a NULL check, and handle both the NULL and non-NULL cases), and so forth. What I'd really like to see, though, is a language with data-parameterized "types" that unfold (as a last resort, when the concrete type cannot be inferred at compile-time) into runtime data-comparison branches. You should be able to have a raw may-be-valid address with no metadata tag, and then treat that data as having a separate type (a Just or a Nothing) depending on whether the memory address itself is nonzero. You write match clauses; the compiler turns those into NULL checks. (As an aside, such a language would also allow for things like a type for "Float that only has an integer component", or "Even number", or "Positive number", or "Number in the range 5..23", or "String of length 5", or "String that encodes a valid Unicode codepoint sequence", etc. The type system would become your swiss-army knife for expressing invariants.)

johncolanduoni 11y ago
> What I'd really like to see, though, is a language with parameterized typing with hybrid static+runtime enforcement. You should be able to be holding a raw address in memory, and then have that become one of two strong types at the language level depending on the whether that memory address is nonzero—"lifting" the NULL check into the type system, rather than replacing the NULL check with extra type annotation. In Rust, the compiler optimizes pointer types wrapped in Options to nullable pointers. Is this the kind of thing you meant?
- derefr 11y ago
  Possibly—does the Rust FFI allow you to import a C library, writing its foreign-function declarations such that the nullable pointers it returns are transformed into, from Rust's perspective, Option-typed non-nullable pointers, with no runtime memory overhead? And then, if so, does Rust further guarantee that those "virtual" Option-typed pointers can be passed back to the same API "just the way it sent them"? That's sort of what I mean, here: the nullable ptr as the "canonical, machine-level" representation, with the type serving as a language-level abstraction, effectively a "lie" because it has no real effect on the type system, and only serves as syntax sugar for generating runtime checks.

johncolanduoni 11y ago

The problem is NULL does not "fail fast". You could get a NULL pointer as an argument, and then not use that value until much later. Now when the bug shows up, you'll have to work back through your code to even figure out where it came from. > Use of a class type implementing an interface as a "null type" is a very good strategy. Have you ever programmed Objective-C? It uses the null object pattern (called "nil") for all Objective-C objects. This makes it really easy to put big logic errors into your code, since calling upon a nil will just give you more nil. So now you have to wait even longer until a fatal error pops up, and then find where on earth the original nil came from. > NULL and nullptr are convenient ways to do that without a ton of extra code. How do you figure? If malloc can return NULL and you want a sane error message, you need to write a NULL check whenever you do a malloc. Otherwise, your user will just get a segfault, instead of a notification that it was memory they ran out of. So why not do that by default, and for the few places where you have prepared yourself to handle out of memory conditions use a different malloc which returns a value that must be explicitly checked?

jschwartzi 11y ago
Because I don't have a system library that provides a different malloc. It's a property of my API that malloc returns NULL and I don't have any control over that. What I do have control over is what I do with data I get out of malloc. I have to check, somehow, for abnormal conditions. That is a requirement for using the malloc API. I get that this makes it hard to debug in some cases, so I might avoid that by religiously checking for NULL return from malloc and crashing immediately with a sensible error message. In the cases where getting a NULL return is not an error, like if I wrote a program to use as much heap as it can, I would ignore the NULL and proceed with the resources I can actually get.
- johncolanduoni 11y ago
  What stops you from writing an API on top of the system malloc that has two malloc functions like I described above, with return types that have unambiguous nullity? Then your religious NULL checks won't be duplicated (and you won't be able to stray from the righteous path and forget any).

mikeash 11y ago

An optional type's "none" value represents the same properties that NULL does. The difference is that the optional type better expresses what's going on. NULL breaks the type system. Now every pointer type is actually two types: the normal type, and the NULL type. You can dereference or call methods on one, but not the other, and only a runtime check will tell you what you're dealing with. Any time you see: Object * You cannot read that as "pointer to Object." You have to read it as "pointer to Object, or NULL." This causes three big problems. One is that it's way too easy to forget to check for NULL. The second, related problem is that without a way to express a non-nullable pointer type, you have to rely on documentation and convention to express the difference. The third is that since only pointer types are nullable, you have to reinvent the wheel any time you you want to express the concept of "null" for a non-pointer type. To illustrate, you point out malloc which returns NULL when it runs out of address space. What's the appropriate return type? void * Now let's say you had some other memory allocation call which could never return failure to the caller. Maybe it aborts execution on failure, or maybe it retries continually until memory becomes available. What's the appropriate return type? void * Oops. Same problem with parameters. Here's a function: void f(void *ptr) Are you allowed to pass NULL? Who knows, go RTFM. If the answer is "no," what happens if you pass NULL anyway? The compiler certainly won't catch it, and who knows what will happen. NULL is "just a tool" but it's not a very good tool. It breaks the type system in a fundamental and painful way. Edit: whew, HN's markup makes it kind of hard to talk about C pointers!

jschwartzi 11y ago
Right, but in malloc's case, it needs to express something about the fundamental type of all software, which is memory. There is nothing more fundamental to the software system. When you run out of memory and fail to check for that condition, the only right and proper thing to do is crash. NULL is a handy way to enforce that fundamentally, in that dereferencing a null pointer guarantees a crash. Part of programming is about memory management. You can hide it with a sophisticated compiler but it's always going to be there, and there are always going to be two types of memory: that which is available, and that which is not.
- 11y ago