5 ms·

Until the virtual address space gets so big that there are no bits left at the top that are "free". As soon as you have a 64 bit virtual address space, pointer

Until the virtual address space gets so big that there are no bits left at the top that are "free". As soon as you have a 64 bit virtual address space, pointer tagging the top bits no longer works. Your program is now broken on the new CPU.

devit 5y ago
Of course competent programmers using the top K bits of the pointer would make sure that the kernel and memory allocator operate so that they don't allocate memory outside a (64 - K)-bit address space, regardless of what the CPU supports (e.g. Linux mmap as one would expect won't allocate beyond 2^48 unless explicitly requested). And obviously any intelligent CPU architect, if they make a CPU that masks the upper bits, they would make the number of bits that gets masked configurable per-process and not dependent on the maximum virtual address space supported by the CPU.
- celeritascelery 5y ago
  > Of course competent programmers ... And obviously any intelligent CPU architect Interesting to see the future is so "obvious" to you. You are correct however, that this is not a problem at the present because no architecture uses more then 48-bits of address space in practice. And since this is such a common practice now, there will have to allowances made in the future for when the address space is expanded.
  - vlovich123 5y ago
    As a sibling to you noted, Ice Lake has bumped this from 48 bits to 57 bits to support 5-level paging [1]. So the "in practice" appears to already by out of date. This was known publicly as early as 2017 (albeit this is the first I've heard of it). That being said, I don't know why the increase from 256TB to 128PB is at all useful but it's enabled in my Arch Linux kernel on a 32GB laptop, so clearly adopted by the broader ecosystem even outside whatever niche use-case it's for. That being said, the kernel maintains backward compatability by restricting the allocation address space it hands out by default. Userspace needs to explicitly ask for the higher bits [2], so it's probably OK to just continue as-is assuming the 48-bit limit in practice. [1] https://en.wikipedia.org/wiki/Intel_5-level_paging https://en.wikipedia.org/wiki/Intel_5-level_paging [2] https://www.kernel.org/doc/html/latest/x86/x86_64/5level-paging.html#:~:text=On%20x86%2C%205%2Dlevel%20paging,ready%20to%20handle%20wide%20addresses.&text=Specifying%20high%20hint%20address%20on,from%2047%2Dbit%20address%20space https://www.kernel.org/doc/html/latest/x86/x86_64/5level-pag....
- zozbot234 5y ago
  > And obviously any intelligent CPU architect, if they make a CPU that masks the upper bits, they would make the number of bits that gets masked configurable per-process x86-64 does this by requiring all accesses to be correctly sign-extended, which means that valid virtual addresses are unique and there is full forward compatibility even with an extended virtual address space. The "number of bits that get masked" is a pure system/ABI concern, the CPU support only defines a maximum.

devit 5y ago

Of course competent programmers using the top K bits of the pointer would make sure that the kernel and memory allocator operate so that they don't allocate memory outside a (64 - K)-bit address space, regardless of what the CPU supports (e.g. Linux mmap as one would expect won't allocate beyond 2^48 unless explicitly requested). And obviously any intelligent CPU architect, if they make a CPU that masks the upper bits, they would make the number of bits that gets masked configurable per-process and not dependent on the maximum virtual address space supported by the CPU.

celeritascelery 5y ago
> Of course competent programmers ... And obviously any intelligent CPU architect Interesting to see the future is so "obvious" to you. You are correct however, that this is not a problem at the present because no architecture uses more then 48-bits of address space in practice. And since this is such a common practice now, there will have to allowances made in the future for when the address space is expanded.
- vlovich123 5y ago
  As a sibling to you noted, Ice Lake has bumped this from 48 bits to 57 bits to support 5-level paging [1]. So the "in practice" appears to already by out of date. This was known publicly as early as 2017 (albeit this is the first I've heard of it). That being said, I don't know why the increase from 256TB to 128PB is at all useful but it's enabled in my Arch Linux kernel on a 32GB laptop, so clearly adopted by the broader ecosystem even outside whatever niche use-case it's for. That being said, the kernel maintains backward compatability by restricting the allocation address space it hands out by default. Userspace needs to explicitly ask for the higher bits [2], so it's probably OK to just continue as-is assuming the 48-bit limit in practice. [1] https://en.wikipedia.org/wiki/Intel_5-level_paging https://en.wikipedia.org/wiki/Intel_5-level_paging [2] https://www.kernel.org/doc/html/latest/x86/x86_64/5level-paging.html#:~:text=On%20x86%2C%205%2Dlevel%20paging,ready%20to%20handle%20wide%20addresses.&text=Specifying%20high%20hint%20address%20on,from%2047%2Dbit%20address%20space https://www.kernel.org/doc/html/latest/x86/x86_64/5level-pag....
zozbot234 5y ago
> And obviously any intelligent CPU architect, if they make a CPU that masks the upper bits, they would make the number of bits that gets masked configurable per-process x86-64 does this by requiring all accesses to be correctly sign-extended, which means that valid virtual addresses are unique and there is full forward compatibility even with an extended virtual address space. The "number of bits that get masked" is a pure system/ABI concern, the CPU support only defines a maximum.