6 ms·

I can buy that dereferencing null is a special case, but why is 0x2 special? Is 0x20 also special? What about 0x20000? Are the invalid non-null pointer values l

I can buy that dereferencing null is a special case, but why is 0x2 special? Is 0x20 also special? What about 0x20000? Are the invalid non-null pointer values listed in a reference somewhere? If 0x2 is an invalid pointer, what do I do if my microcontroller has a hardware register at 0x2?

jeroenhd 4y ago
On many platforms, the zero page is set up so access to it will always segfault. This isn't a language guarantee, but it's a guarantee in most modern operating systems (Linux, FreeBSD, Windows). This is set up for pointers all the way up to the end of the first page. On Windows and Linux this is the first 4KiB so range 0x0000 up to 0x1000, unless large pages are on (then it's even more). On macOS in x64 this is the entire 4GiB memory space, probably a method to help developers port their 32-bit software to x64. I don't know what the zero page size on ARM is. If your microcontroller doesn't have this guarantee, you can't make use of this feature.
- rom-antics 4y ago
  That's a guarantee on the level of the hardware/OS, but hardware semantics are not the same as language/compiler semantics. Even if according to the source code you're dereferencing a pointer value 0x0 or 0x2, that doesn't mean the compiler-emitted machine code will end up telling the hardware to do the same. Remember this gem? https://kristerw.blogspot.com/2017/09/why-undefined-behavior-may-call-never.html https://kristerw.blogspot.com/2017/09/why-undefined-behavior... Once you trigger UB, all bets are off and your code could do anything. A segfault just means you spun the roulette wheel, bet it all on red, and got lucky your house wasn't bulldozed. Zig also uses LLVM under the hood, right? So it's subject to these same semantics. An LLVM pointer value cannot legally contain arbitrary non-null non-pointer integers such as 0x2. That's a dead giveaway of UB. And I doubt the emitted Zig code safety-checks every pointer dereference for a value less than 0x1000 before performing the dereference.
  - kristoff_it 4y ago
    > An LLVM pointer value cannot legally contain arbitrary non-null non-pointer integers such as 0x2. 0x2 is a perfectly valid pointer value, it just happens to never be a good virtual memory address on modern systems where virtual memory is setup by the usual OSs, hence the fact that you can rely on it segfaulting.
  - jeroenhd 4y ago
    The semantics are actually operating system and even compiler flag dependent. On macOS you can choose the size of your zero page during build. The numbers I've listed are just the defaults. Zig UB is not C UB. There is an entire language built on top of it. Just because something behaves a certain way in C, doesn't mean the same thing is true in Zig. Zig is no longer a code generator for C, it has switched to a self hosted compiler a while back. In fact, the language is rapidly progressing to the point where LLVM is a mere optional dependency. I don't know the semantics around LLVM pointers. I don't see why 0x2 would be invalid, there are plenty of platforms programmed in C(++) that have a flat memory model. It would be quite painful to have a microcontroller where you can't send data to the output pin because LLVM decided that 2 is invalid (but 0 isn't). I've never seen LLVM complain about invalid dereferencing, though, it always ends up doing what the compiler tells it to do as far as I can tell. Zig pointers will definitely cause UB but most Zig code shouldn't need them. Slices are actually bound checked and should probably be preferred in most cases of pointer arithmetic. Simple pointers can't be increased or decremented so you need to manually go through @intToPtr if you want to do real pointer arithmetic, which is quite unusable. I haven't used Zig much so I don't know how many Zig semantics are copies of C semantics and how many are translated by the Zig frontend. However, "this is a bad/undefined thing in C so it must be a bad/undefined thing in Zig" is simply not true.
    - rom-antics 4y ago
      I know Zig is not C, that's why I specifically mentioned LLVM. It's fine if Zig has different opinions about UB than LLVM does, but in that case ReleaseSafe builds should not use LLVM, not even optionally. If Zig says some operation is defined, but LLVM says it's undefined, well, LLVM is the one optimizing code so it's LLVM's invariants that matter. Right now it looks like Zig is playing fast and loose with correctness, shoving everything through LLVM but not respecting LLVM's invariants. And hey, if something is observed to segfault under some conditions today on the current version of LLVM, we'll just say segfaults are guaranteed. It's disappointing to see.
- avgcorrection 4y ago
  > On many platforms, the zero page is set up so access to it will always segfault. This isn't a language guarantee, but it's a guarantee in most modern operating systems (Linux, FreeBSD, Windows). This is set up for pointers all the way up to the end of the first page. Then I guess it could be a language guarantee if Zig only supports/targets those platforms. However, considering how low-level Zig is, I doubt that that is the case.
throwawaymaths 4y ago
> I can buy that dereferencing null is a special case, but why is 0x2 special? It isn't, in the general case. But JavaScript engines do some dark magic with pointer packing / NaN boxing as a performance optimization (most things in the VM are single words, passing around a single word is usually way cheaper than full unboxing), and I suspect bun in occasionally running into issues where it gets returned something from the JS engine which it thinks is a pointer but actually it's a packed, special value. This is a logic error, that turns into a weird memory issue at the abi boundary, not a memory safety issue.
cultureswitch 4y ago
Not even 0x0 in the general case it could a legit address.

jeroenhd 4y ago

On many platforms, the zero page is set up so access to it will always segfault. This isn't a language guarantee, but it's a guarantee in most modern operating systems (Linux, FreeBSD, Windows). This is set up for pointers all the way up to the end of the first page. On Windows and Linux this is the first 4KiB so range 0x0000 up to 0x1000, unless large pages are on (then it's even more). On macOS in x64 this is the entire 4GiB memory space, probably a method to help developers port their 32-bit software to x64. I don't know what the zero page size on ARM is. If your microcontroller doesn't have this guarantee, you can't make use of this feature.

rom-antics 4y ago
That's a guarantee on the level of the hardware/OS, but hardware semantics are not the same as language/compiler semantics. Even if according to the source code you're dereferencing a pointer value 0x0 or 0x2, that doesn't mean the compiler-emitted machine code will end up telling the hardware to do the same. Remember this gem? https://kristerw.blogspot.com/2017/09/why-undefined-behavior-may-call-never.html https://kristerw.blogspot.com/2017/09/why-undefined-behavior... Once you trigger UB, all bets are off and your code could do anything. A segfault just means you spun the roulette wheel, bet it all on red, and got lucky your house wasn't bulldozed. Zig also uses LLVM under the hood, right? So it's subject to these same semantics. An LLVM pointer value cannot legally contain arbitrary non-null non-pointer integers such as 0x2. That's a dead giveaway of UB. And I doubt the emitted Zig code safety-checks every pointer dereference for a value less than 0x1000 before performing the dereference.
- kristoff_it 4y ago
  > An LLVM pointer value cannot legally contain arbitrary non-null non-pointer integers such as 0x2. 0x2 is a perfectly valid pointer value, it just happens to never be a good virtual memory address on modern systems where virtual memory is setup by the usual OSs, hence the fact that you can rely on it segfaulting.
- jeroenhd 4y ago
  The semantics are actually operating system and even compiler flag dependent. On macOS you can choose the size of your zero page during build. The numbers I've listed are just the defaults. Zig UB is not C UB. There is an entire language built on top of it. Just because something behaves a certain way in C, doesn't mean the same thing is true in Zig. Zig is no longer a code generator for C, it has switched to a self hosted compiler a while back. In fact, the language is rapidly progressing to the point where LLVM is a mere optional dependency. I don't know the semantics around LLVM pointers. I don't see why 0x2 would be invalid, there are plenty of platforms programmed in C(++) that have a flat memory model. It would be quite painful to have a microcontroller where you can't send data to the output pin because LLVM decided that 2 is invalid (but 0 isn't). I've never seen LLVM complain about invalid dereferencing, though, it always ends up doing what the compiler tells it to do as far as I can tell. Zig pointers will definitely cause UB but most Zig code shouldn't need them. Slices are actually bound checked and should probably be preferred in most cases of pointer arithmetic. Simple pointers can't be increased or decremented so you need to manually go through @intToPtr if you want to do real pointer arithmetic, which is quite unusable. I haven't used Zig much so I don't know how many Zig semantics are copies of C semantics and how many are translated by the Zig frontend. However, "this is a bad/undefined thing in C so it must be a bad/undefined thing in Zig" is simply not true.
  - rom-antics 4y ago
    I know Zig is not C, that's why I specifically mentioned LLVM. It's fine if Zig has different opinions about UB than LLVM does, but in that case ReleaseSafe builds should not use LLVM, not even optionally. If Zig says some operation is defined, but LLVM says it's undefined, well, LLVM is the one optimizing code so it's LLVM's invariants that matter. Right now it looks like Zig is playing fast and loose with correctness, shoving everything through LLVM but not respecting LLVM's invariants. And hey, if something is observed to segfault under some conditions today on the current version of LLVM, we'll just say segfaults are guaranteed. It's disappointing to see.
avgcorrection 4y ago
> On many platforms, the zero page is set up so access to it will always segfault. This isn't a language guarantee, but it's a guarantee in most modern operating systems (Linux, FreeBSD, Windows). This is set up for pointers all the way up to the end of the first page. Then I guess it could be a language guarantee if Zig only supports/targets those platforms. However, considering how low-level Zig is, I doubt that that is the case.

throwawaymaths 4y ago

> I can buy that dereferencing null is a special case, but why is 0x2 special? It isn't, in the general case. But JavaScript engines do some dark magic with pointer packing / NaN boxing as a performance optimization (most things in the VM are single words, passing around a single word is usually way cheaper than full unboxing), and I suspect bun in occasionally running into issues where it gets returned something from the JS engine which it thinks is a pointer but actually it's a packed, special value. This is a logic error, that turns into a weird memory issue at the abi boundary, not a memory safety issue.

cultureswitch 4y ago

Not even 0x0 in the general case it could a legit address.