3 ms·

isn't like, memory tagging explicitly forbidden in x86-64 ? iirc it was used in lisp compilers/interpreters to speed up object unboxing (as in answering the qu

by znpy 6y ago

isn't like, memory tagging explicitly forbidden in x86-64 ? iirc it was used in lisp compilers/interpreters to speed up object unboxing (as in answering the question: "what class does the object pointed by this pointer belong to?")

colejohnson66 6y ago
Correct. x86-64 mandates “canonical” addresses where all the upper bits must be the same in order to dereference a pointer.[0] So on a processor with 48 physical address lines, the upper 16 must be either all set or all unset. This is obviously to prevent people using those upper bits to store extra data. Of course, this doesn’t stop you from being able to[a], it just makes it harder. [a]: The processor doesn’t know the “type” of a register’s value; It’s just bits. It could be a pointer, integer, floating point, etc. until you tell the processor to do something with it. So one could obviously use those extra bits to store data, then when a dereference is needed, store those bits in another register, dereference, then put the bits back. But that’s a lot of work to save a byte or two. [0]: Intel SDM Volume 1 §3.3.7.1: > In 64-bit mode, an address is considered to be in canonical form if address bits 63 through to the most-significant implemented bit by the microarchitecture are set to either all ones or all zeros.
- staticassertion 6y ago
  > But that’s a lot of work to save a byte or two. Is it? It's something like two xors on a register. If that's a byte or two per pointer (ie: it's pointer metadata that you'd have to store anyway, which I assume is the use case) that's actually a pretty significant win, and could help reduce cache pressure, thereby buying you back the performance (and then some). Addresses are also XOR'd all the time with a secret as a mitigation, I don't know that there's much of an impact at all.
imglorp 6y ago
Absolutely, or in the low bits of word aligned pointers as someone else mentioned above. Emacs Lisp is an example; here's its tagging apparatus. https://git.savannah.gnu.org/cgit/emacs.git/tree/src/lisp.h#n207 https://git.savannah.gnu.org/cgit/emacs.git/tree/src/lisp.h#...
StillBored 6y ago
And common javascript engines. Turns out a lot of interpreters hide bits away in what they precieve to be unused address bits. Its one of those lessons that everyone seems to be constantly relearning. Address spaces grow. One of my first "real" jobs, when this topic came up a very senior person said something to the effect. We continue to find ways to use up one of those bits roughly every year. Moving from 32-64 bit VA's is going to get beyond the end of my career but someone will have to deal with it in the future. So its a little slower than 1 bit a year, but everyone is adding a few more bits (52/56/etc) because there are actual commercial computer systems with a few tens of TB of ram today. If persistent memory ever becomes a thing, someone _WILL_ want to mmap their multi PB storage cluster, and then there will be a scramble for more address bits.
- guenthert 6y ago
  Value tagging may (and often does) use the lower bits to distinguish pointers from other values (integer, boolean, character, etc.) as objects are memory aligned. In those schemes the growth of the address space doesn't matter.
  - monocasa 6y ago
    You can use the upper bits too. It's fairly common to use the top bit to discriminate between a pointer or an integer, since you can just test if the value is negative, which can be free on some architectures.
fulafel 6y ago
No, like on most other archs, people use tagged pointers in lisp and other runtimes on amd64 all the time. But the machine (like most ISAs) doesn't ignore any bits so you have to clear the tag bits before dereferencing. You may be thinking of the fact that a full 64 bit VA space isn't supported in current implementations. But this has no impact on use of tagging compared to most other archs.

by znpy 6y ago

colejohnson66 6y ago

Correct. x86-64 mandates “canonical” addresses where all the upper bits must be the same in order to dereference a pointer.[0] So on a processor with 48 physical address lines, the upper 16 must be either all set or all unset. This is obviously to prevent people using those upper bits to store extra data. Of course, this doesn’t stop you from being able to[a], it just makes it harder. [a]: The processor doesn’t know the “type” of a register’s value; It’s just bits. It could be a pointer, integer, floating point, etc. until you tell the processor to do something with it. So one could obviously use those extra bits to store data, then when a dereference is needed, store those bits in another register, dereference, then put the bits back. But that’s a lot of work to save a byte or two. [0]: Intel SDM Volume 1 §3.3.7.1: > In 64-bit mode, an address is considered to be in canonical form if address bits 63 through to the most-significant implemented bit by the microarchitecture are set to either all ones or all zeros.

staticassertion 6y ago
> But that’s a lot of work to save a byte or two. Is it? It's something like two xors on a register. If that's a byte or two per pointer (ie: it's pointer metadata that you'd have to store anyway, which I assume is the use case) that's actually a pretty significant win, and could help reduce cache pressure, thereby buying you back the performance (and then some). Addresses are also XOR'd all the time with a secret as a mitigation, I don't know that there's much of an impact at all.

imglorp 6y ago

Absolutely, or in the low bits of word aligned pointers as someone else mentioned above. Emacs Lisp is an example; here's its tagging apparatus. https://git.savannah.gnu.org/cgit/emacs.git/tree/src/lisp.h#n207 https://git.savannah.gnu.org/cgit/emacs.git/tree/src/lisp.h#...

StillBored 6y ago

And common javascript engines. Turns out a lot of interpreters hide bits away in what they precieve to be unused address bits. Its one of those lessons that everyone seems to be constantly relearning. Address spaces grow. One of my first "real" jobs, when this topic came up a very senior person said something to the effect. We continue to find ways to use up one of those bits roughly every year. Moving from 32-64 bit VA's is going to get beyond the end of my career but someone will have to deal with it in the future. So its a little slower than 1 bit a year, but everyone is adding a few more bits (52/56/etc) because there are actual commercial computer systems with a few tens of TB of ram today. If persistent memory ever becomes a thing, someone _WILL_ want to mmap their multi PB storage cluster, and then there will be a scramble for more address bits.

guenthert 6y ago
Value tagging may (and often does) use the lower bits to distinguish pointers from other values (integer, boolean, character, etc.) as objects are memory aligned. In those schemes the growth of the address space doesn't matter.
- monocasa 6y ago
  You can use the upper bits too. It's fairly common to use the top bit to discriminate between a pointer or an integer, since you can just test if the value is negative, which can be free on some architectures.

fulafel 6y ago

No, like on most other archs, people use tagged pointers in lisp and other runtimes on amd64 all the time. But the machine (like most ISAs) doesn't ignore any bits so you have to clear the tag bits before dereferencing. You may be thinking of the fact that a full 64 bit VA space isn't supported in current implementations. But this has no impact on use of tagging compared to most other archs.