14 ms·

If you're serious about smuggling data in pointers, there's probably a lot of mileage on the other end of the pointer if you're writing 64-bit code. While 32 bi

If you're serious about smuggling data in pointers, there's probably a lot of mileage on the other end of the pointer if you're writing 64-bit code. While 32 bits can address 4Gb, you only need another 8 bits (say) to get a terabyte, leaving maybe 24 bits to cram with whatever you feel like.

derleth 15y ago
Macintosh programmers used to do this. Used to. http://en.wikipedia.org/wiki/Mac_OS_memory_management#32-bit_clean http://en.wikipedia.org/wiki/Mac_OS_memory_management#32-bit...
nitrogen 15y ago
It's probably not a good idea to rely on this. Though x86-64 is (currently?) limited to a 48-bit virtual address space, it's designed so that the upper 17 bits must be all 0 or all 1, and those bits may be used in the future. https://secure.wikimedia.org/wikipedia/en/wiki/X86-64#Virtual_address_space_details https://secure.wikimedia.org/wikipedia/en/wiki/X86-64#Virtua...
- rvkennedy 15y ago
  A good idea? I think it's nuts! I strongly recommend that nobody ever do this.
  - vidarh 15y ago
    It'd be ok to do for a JIT, assuming the JIT knows the systems it targets can deal with it and/or has a fallback strategy for systems that don't. But yes, not for ahead of time compiled code...
dubya 15y ago
The enormity of 64-bit pointers is something Don Knuth has complained about because it wastes a lot of memory for programs that don't need that much address space. And it actually matters for some of the stuff he writes. The old Motorola had an addressing mode that used 16 bit relative pointers. Does Intel have anything similar for x64?
- dmpk2k 15y ago
  The JVM takes advantage of this: http://wikis.sun.com/display/HotSpotInternals/CompressedOops http://wikis.sun.com/display/HotSpotInternals/CompressedOops
- dubya 15y ago
  Replying to myself :( The mode was called PC relative addressing, and resulted in smaller, faster code on the 68000, but things had to be written in 32k chunks. I seem to recall that it was required for the original Macintosh before they wrote a proper program loader. Anyhow, some of Knuth's code (hiss BDD package at least) has hacks to reduce the size of pointers.
- BruceJillis 15y ago
  Came here to make a remark about Knuth. Indeed, in TACP vol 1, fascicle 1 (the new MMIX design) he talks about being able to address memory in chunks larger than 8 bits and specifically remarks about the unused bits: "we will find precious use for them later". :)
olliej 15y ago
Using tags in the high bits is exactly what caused all those problems for software written in the 80s and early 90s. The assumption that your software will have been updated before anyone has such "ludicrous" amounts of memory is not safe. Low bits seem to be generally safe as a) you can easily control pointer alignment in your own code, and b) most system's alignment requirements are increasing not decreasing. Some architectures can't even access unaligned pointers. Even excluding large ram increases, presumably the current (effective) 48bit address space limitation on x86_64 is likely to go away, and then ASLR is going to make it hard to guarantee that your high bits are clear.

derleth 15y ago

Macintosh programmers used to do this. Used to. http://en.wikipedia.org/wiki/Mac_OS_memory_management#32-bit_clean http://en.wikipedia.org/wiki/Mac_OS_memory_management#32-bit...

nitrogen 15y ago

It's probably not a good idea to rely on this. Though x86-64 is (currently?) limited to a 48-bit virtual address space, it's designed so that the upper 17 bits must be all 0 or all 1, and those bits may be used in the future. https://secure.wikimedia.org/wikipedia/en/wiki/X86-64#Virtual_address_space_details https://secure.wikimedia.org/wikipedia/en/wiki/X86-64#Virtua...

rvkennedy 15y ago
A good idea? I think it's nuts! I strongly recommend that nobody ever do this.
- vidarh 15y ago
  It'd be ok to do for a JIT, assuming the JIT knows the systems it targets can deal with it and/or has a fallback strategy for systems that don't. But yes, not for ahead of time compiled code...

dubya 15y ago

The enormity of 64-bit pointers is something Don Knuth has complained about because it wastes a lot of memory for programs that don't need that much address space. And it actually matters for some of the stuff he writes. The old Motorola had an addressing mode that used 16 bit relative pointers. Does Intel have anything similar for x64?

dmpk2k 15y ago
The JVM takes advantage of this: http://wikis.sun.com/display/HotSpotInternals/CompressedOops http://wikis.sun.com/display/HotSpotInternals/CompressedOops
dubya 15y ago
Replying to myself :( The mode was called PC relative addressing, and resulted in smaller, faster code on the 68000, but things had to be written in 32k chunks. I seem to recall that it was required for the original Macintosh before they wrote a proper program loader. Anyhow, some of Knuth's code (hiss BDD package at least) has hacks to reduce the size of pointers.
BruceJillis 15y ago
Came here to make a remark about Knuth. Indeed, in TACP vol 1, fascicle 1 (the new MMIX design) he talks about being able to address memory in chunks larger than 8 bits and specifically remarks about the unused bits: "we will find precious use for them later". :)

olliej 15y ago

Using tags in the high bits is exactly what caused all those problems for software written in the 80s and early 90s. The assumption that your software will have been updated before anyone has such "ludicrous" amounts of memory is not safe. Low bits seem to be generally safe as a) you can easily control pointer alignment in your own code, and b) most system's alignment requirements are increasing not decreasing. Some architectures can't even access unaligned pointers. Even excluding large ram increases, presumably the current (effective) 48bit address space limitation on x86_64 is likely to go away, and then ASLR is going to make it hard to guarantee that your high bits are clear.