5 ms·
Years ago, someone said "32 bit addresses! Thats huge! Lets use the top few bits for other stuff, like gate A20 and select lines of hardware". That came to b
by londons_explore 5y ago
Years ago, someone said "32 bit addresses! Thats huge! Lets use the top few bits for other stuff, like gate A20 and select lines of hardware".
That came to bite people in the form of the "3GB hole", the "PCI hole", etc. And those hacks were painful for lots of people.
I feel that by reusing address bits for other purposes in a non-flexible way like this, we're just repeating the mistakes of history. After all, there are 44 zettabytes of data in the world (in 2020), and addressing that is already beyond what a 64 bit number can do!
And one day, we'll have that much storage in your hand - your phone has about 2^80 atoms in it, so storing 2^64 bytes in there is totally physically possible.
- pjmlp 5y agoHopefully by then the languages whose reason for pointer tagging exists in first place, won't matter as much as they still do today.
- karatinversion 5y agoPointers are a core part of the CPU memory model on both ARM and x64, so they aren't going away anytime soon; there's no reason compilers, JITs or runtimes couldn't use pointer tagging, just the same as C programmers.
- pjmlp 5y agoTrue, but the way they are exposed to the programmers matter. Languages like Modula-2 or Ada, to use two C like languages, don't suffer from pointer abuse as much as C, because they provide better abstractions, thus raw pointer tricks aren't as dangerous to the whole system stability. While JIT and runtimes could make use of tagging as well, Lisp and Ada machines kind of proved the point that with memory safe runtimes, the CPU specialy extensions aren't as useful as general purpose CPUs hence why they moved away from such solutions. So there is a certain irony that we are now going back into C Machines as the ultimate mitigation.
- saagarjha 5y agoLisp and Ada machines happened to be very slow in practice. Turns out people prefer CPU extensions to make their C code fast rather than compiling it to a more exotic architecture.
- pjmlp 5y agoI missed the point, Lisp and Ada machines were never designed to run C code, other than a curiosity.
- saagarjha 5y agoWell, the two points I was making were 1. people want to run C code anyways, so that's not really an option for general-purpose use and 2. Lisp and Ada machines ended up being slow for Lisp and Ada, which makes them pretty unattractive.
- cmrdporcupine 5y agoGiven its longevity and installed base, I see no reasonable possibility of C/C++ going away or being significantly curtailed in the next half century. C is 50 years old this year. And it's everywhere It's more likely we'll be (or my kids will be) involved in harm reduction around these systems, rather than outright replacement.
- praptak 5y agoWho knows, maybe we'll have a cheap (semi-)automatic rewrite from C to a memory-safe language by then.
- londons_explore 5y agoLook to other industries. There are plenty of standard things that are suboptimal, but still used because they're so standard. For example, the English language has plenty of inefficiencies and inconsistencies that date back over 1000 years. But to fix them would be too much work for too little gain.
- pjmlp 5y agoCobol and Fortran are still around, they even support modules, some form of generics and OOP, yet you don't see many people jump of joy writing new libraries in them.
- cmrdporcupine 5y agoWhereas people still jump for joy at writing new libraries in C++ and it's coming up on 40 years old. C++11 saved the language from dying off. Somehow it's my favourite language to work in (though mainly because nobody is paying me to work in Rust)
- pjmlp 5y agoWe are talking mostly about C here, but in any case, I don't jump of joy when looking at code that depends on SFINAE, ADL and CTAD.
- rwmj 5y agoI don't think C is going away any time soon, and even if it did, C reflects a common model of how hardware works that is shared by plenty of other higher level languages. Also GCs use pointer tagging.
- pjmlp 5y agoThe irony of C Machines.
- saagarjha 5y agoYou mean most languages that run on virtual machines today, right? Java, JavaScript, C#, Lua, …
- pjmlp 5y agoLLVM, WASM,....
- tawaypol 5y agoLLVM is as much a virtual machine as the "C Virtual Machine" from the spec is. To my knowledge there are no LLVM IR interpreters.
- The_rationalist 5y agohttps://llvm.org/docs/tutorial/BuildingAJIT1.html https://llvm.org/docs/tutorial/BuildingAJIT1.html
- pjmlp 5y agowatchOS uses an Apple flavour of LLVM bitcode as executable format. GraalVM also supports LLVM IR.
- saagarjha 5y agowatchOS does not support executing LLVM IR, or runs normal ARM code with a somewhat tweaked ABI that’s generated from Bitcode.
- pjmlp 5y agoExecutable format does not mean it is executed directly, rather the format used for the executable files shipped into the store, which then does the AOT compilation for the specific version of watchOS. Better now?
- 5y ago
- colejohnson66 5y agoThe whole reason x86-64 required “canonical”[a] addresses in the first place was to prevent people using them for this purpose. Sure, this proposal allows applications to know how many bits they can work with, but what happens when an application developer writes `assert(freeBits >= 7)` when only 6 are available in the future? [a] a “canonical” address is one where bits 63:MaxPhysAddr are identical. So on a processor supporting 48 address lines, bits 63:48 must be identical (either all clear or all set). Attempting reads or writes with non canonical addresses raises a #GP fault.
- zozbot234 5y ago> Sure, this proposal allows applications to know how many bits they can work with That is entirely orthogonal to this "upper bits ignore" feature. Ideally, a process would be able to set any reasonable amount of upper bits as "reserved for tagging", and the system allocator and kernel would then simply not require it to work with user virtual addresses that involve those upper bits. But upper bits can't simply be "ignored" if an app is to be forward-compatible; they still need to be canonicalized whenever external code is called.
- saagarjha 5y agoThen the application does not work on a future processor, that is correct. The person who wrote it, who likely is a JIT engineer, will get a bug report asking for this to be fixed, although most likely they're already aware of the upcoming hardware change and have prepared a different tagging scheme already.
- hyperman1 5y agoThe A20 gate story was something different though, it abused the numeric overflow of 16 bit addresses, not some unused bits in the middle of an address. An x86 adres had a 16 bit segment and a 16 bit offset, with the address being calculated as 16*segment+offset, truncated to 20 bits. Segment F000 was in ROM and 0000 was in RAM, with the low RAM addresses used for different BIOS functionalities. If you use a segment like 0xFF00 , then offset 0 to 0x0FFF correspond to linear addresses 0xFF000 to 0xFFFFF, in ROM. Offset 0x1000 to 0xFFFF correspond to 0x0000 to 0xEFFF, in RAM. This trick meant you can set the segment register only once, and use the offset to read both tables in ROM and use variables in RAM. Of course contemporary BIOS and other software used this trick to shave of a few instructions. You know what happens next. The 80286 has 24 address lines, not 20, so the addresses just above the existing 0xFFFFF = 1MB became valid instead of wrapping around. In the name of backward compatibility, someone found an unused port on the keyboard controller, attached an AND port to it and to A20. If you talk to the keyboard controller, you can choose if you want 16MB ram or the old wraparound behavior. The keyboard controller was of course dog slow, and you have to set and reset the A20 gate the whole time to switch between not triggering BIOS/DOS/TSR bugs and usable upper RAM. This hack is AFAIK still there today in every x86 based PC, even if the gate stays enabled all the time.
- ajross 5y ago> The A20 gate story was something different though, it abused the numeric overflow of 16 bit addresses, not some unused bits in the middle of an address. It wasn't even an abuse. It was a genuine attempt by the board manufacturer (IBM) to address a real backwards compatibility problem with a new CPU part. Obviously as memories grew far beyond 1M and real mode became a part of history (well, and of bootstrap code) it was a giant wart. But it solved a real problem.
- scottlamb 5y ago> I feel that by reusing address bits for other purposes in a non-flexible way like this, we're just repeating the mistakes of history. Those old schemes were permanent machine-wide assumptions on physical addresses. They were essentially saying "32 - N bits is enough for anyone" (on this hardware design). This is on virtual addresses, and the kernel developers want (and it sounds like the Intel feature allows) this to be configured per-process. I think it's totally reasonable for some process to say at runtime "56 bits is enough for me". In fact, it's common for Java code to run with "Compressed Oops" (32-bit pointers to 8-byte-aligned addresses, for a total of 32 GiB of addressable heap). This happens automatically when the configured heap size is sufficiently small. > After all, there are 44 zettabytes of data in the world (in 2020), and addressing that is already beyond what a 64 bit number can do! I don't think it makes sense for one process to be able to mmap all the data in the world. The major page fault latency would be nasty! > And one day, we'll have that much storage in your hand - your phone has about 2^80 atoms in it, so storing 2^64 bytes in there is totally physically possible. That day is pretty far off I think, but even when it happens I'm not sure it makes sense for all virtual pointers to be >=64-bit. A couple reasons it might not: * Most programs (depending somewhat on programming language) use a lot of their memory for pointers, so it seems wasteful. * I assume this hypothetical future device will still have slower and faster bytes to access. It may still make more sense to have different APIs for the fast stuff and the slow stuff. mmap's limitation of stalling the thread while something is paged in is a real problem today, and I don't know if that would get any better. Likewise error handling.