4 ms·

> The "niche" in this case is the realization that strings are unlikely to be exabytes in size,... That’s what we say now, but a few dozen years down the line?

> The "niche" in this case is the realization that strings are unlikely to be exabytes in size,... That’s what we say now, but a few dozen years down the line? Old Mac OS versions (pre X) would store extra bits of information in the MSBs of a pointer. “32 bit clean” Mac OS had to be created to allow usage on computers with more than a few dozen megabytes. That was an all too common problem when multiple gigabytes of RAM became commonplace. Granted, I highly doubt we’ll see exabyte level string lengths, but you never know. Fun fact: x86-64 mandates pointers be in “canonical” form where all unused bits of the address are the same as the MSB. So on a processor that has 48 address lines, the upper 16 must match the 48th bit. If you try to access memory with a “non canonical” pointer, the processor will throw an exception. This is because they don’t want people using them for flag bits and having a repeat of the “unclean” software problem. TL;DR: I agree, but never say never

KMag 6y ago
And it's too bad that most kernels (including Linux and Windows) put userspace in the low half of the address space. In the upper half of a 48-bit address space (or even 51-bit), all valid pointers, if interpreted as IEEE-754 doubles, are NaN, so you get NaN boxing without performing any arithmetic. (Though, if you want NaN-boxed nullable pointers, you'll need to pick a special value for nullptr.)
- colejohnson66 6y ago
  Why are you treating pointers as floating point numbers?
  - KMag 6y ago
    Pointers aren't being treated as doubles. There are 2**53-2 bit patterns for NaN values, and most FPUs (including x87 and the Aarch64 FPU) generate only 2 of those, leaving 2**25-4 values available as a variant on tagged pointers called NaN-boxing/nun-boxing. NaN-boxing / nun-boxing is a common optimization, as seen in Firefox's SpiderMonkey, Safari's JSC, LuaJIT, etc. If all of your valid pointers are at the top of the address space, you can use 0x0 as the constant you either add/subtract or xor with your values to box/un-box. In other words, Nan-boxing/unboxing becomes a no-op.
lpapez 6y ago
I think we have quite some before us until the need for these high bits becomes reality :) Now that you mention pointers, Apple is using unused bits in their values for signing and authentication purposes https://googleprojectzero.blogspot.com/2019/02/examining-pointer-authentication-on.html?m=1 https://googleprojectzero.blogspot.com/2019/02/examining-poi... Pretty neat in my opinion
Dylan16807 6y ago
There are two problems with using high pointer bits as flags. The small problem is that your program will be limited in memory space. That's usually okay. If you want a vast expansion of memory space you probably need to start using different algorithms anyway. The big problem is that your program depends on the CPU ignoring those bits, and now it can't run at all on updated hardware. As you noted, x86-64 solves this problem by forcing programs to clean up pointers before actually using them.

KMag 6y ago

And it's too bad that most kernels (including Linux and Windows) put userspace in the low half of the address space. In the upper half of a 48-bit address space (or even 51-bit), all valid pointers, if interpreted as IEEE-754 doubles, are NaN, so you get NaN boxing without performing any arithmetic. (Though, if you want NaN-boxed nullable pointers, you'll need to pick a special value for nullptr.)

colejohnson66 6y ago
Why are you treating pointers as floating point numbers?
- KMag 6y ago
  Pointers aren't being treated as doubles. There are 2**53-2 bit patterns for NaN values, and most FPUs (including x87 and the Aarch64 FPU) generate only 2 of those, leaving 2**25-4 values available as a variant on tagged pointers called NaN-boxing/nun-boxing. NaN-boxing / nun-boxing is a common optimization, as seen in Firefox's SpiderMonkey, Safari's JSC, LuaJIT, etc. If all of your valid pointers are at the top of the address space, you can use 0x0 as the constant you either add/subtract or xor with your values to box/un-box. In other words, Nan-boxing/unboxing becomes a no-op.

lpapez 6y ago

I think we have quite some before us until the need for these high bits becomes reality :) Now that you mention pointers, Apple is using unused bits in their values for signing and authentication purposes https://googleprojectzero.blogspot.com/2019/02/examining-pointer-authentication-on.html?m=1 https://googleprojectzero.blogspot.com/2019/02/examining-poi... Pretty neat in my opinion

Dylan16807 6y ago

There are two problems with using high pointer bits as flags. The small problem is that your program will be limited in memory space. That's usually okay. If you want a vast expansion of memory space you probably need to start using different algorithms anyway. The big problem is that your program depends on the CPU ignoring those bits, and now it can't run at all on updated hardware. As you noted, x86-64 solves this problem by forcing programs to clean up pointers before actually using them.