5 ms·

Let's just hope we don't go through the same thing when we switch to 128-bit. We ought to learn from our mistakes when we do this whole thing again. And yes, b

Let's just hope we don't go through the same thing when we switch to 128-bit. We ought to learn from our mistakes when we do this whole thing again. And yes, before people flare me for this, don't pretend it isn't possible, recall that once upon a time 640K ought to be enough for anybody.

tedunangst 7y ago
Nobody ever thought that, and mindlessly repeating just means we don't learn anything.
eru 7y ago
Why would we want to switch to 128 bit addresses? Do you know how much memory we can address with 64 bit? (64 bit for addresses was arguably a mistake. 48 bit word size would probably have been better, but doesn't sound as cool.) Having said that, 128 bit can make sense for certain calculations. So floating point calculations and GPUs support long registers for some of what they are doing. (And having said that, Google figured out that they don't actually need all that precision when using GPUs for machine learning, and made TPUs with much smaller words.)
- imron 7y ago
  > Why would we want to switch to 128 bit addresses? How else are you going to have more than 16 exabytes of RAM?
  - eru 7y ago
    Exactly, you probably won't. And I don't mean that in an absolute sense. People might very well get up to those orders of magnitude of RAM; but you are unlikely to have that amount of RAM available to a single processor. The extra margin of the 64 bit might help with address space randomization, though.
  - jagger27 7y ago
    Memory banking is the obvious and ancient solution. PAE is another.
    - amluto 7y ago
      PAE is a kludge because the architects needed to add another tiny page table level to keep 32 bits of virtual space, but didn’t really want to add another level. The concept of having more physical address bits than virtual bits is reasonable, although it falls apart a bit with virtualization. The idea of having magic registers that fill themselves in for you and can’t be read at all (such as the PAE PDPTR registers) is a bad idea that unfortunately repeats itself in x86 design. Architects: don’t do this.
- jnwatson 7y ago
  Your modern Intel chip only has 48 addressable virtual address bits. The recent Ice Lake processors support 57 bits. I deal daily with Python processes that have "6.1t" in the "resident" column in top. Hitting the 48-bit 256 TiB limit isn't science fiction to me. I can already hit that with a moderate amount of effort (by memory mapping ~100 large kv-stores). It isn't that I need to do it, but I could imagine that someone does. It is well and proper that Intel rounded up to 64 bits. It should serve us well for the next 40ish years, which is good enough for my professional lifetime at least.
  - eru 7y ago
    Yes, that's partially why I mentioned the 48 bits. It's interesting that you mention hitting that with memory mapping in practice. That is a valid concern.
- dexen 7y ago
  "Why would we want to switch to 128 bit addresses?" - eru, 2019 I am gonna hold onto your quote, for enjoyment and giggles in 2032 :^)
  - dependenttypes 7y ago
    This is only 13 years from now. I really doubt that people will need 16 EiB by then in a single processor for a single process. Plus by that point you will probably have moved to message passing between distributed systems or something.
    - eru 7y ago
      My prediction is what we'll mostly see more of the same, with existing trends continuing. Ie more mobile, and more parallelism on the server side. But yeah, no 16 EiB on a single processor. (Though we might see people memory map crazy amounts, without ever actually accessing all of them, of course.)
- exikyut 7y ago
  The fact that you can put everything, ever into 128-bit means you can easily shove ten thousand 20-terabyte storage devices into a single 128-bit system's memory-mapped I/O space and still have 3 exabytes left over. No more packet-switched serial storage I/O. You now have first-class ability to ask for any byte anywhere, really really really fast. Because I/O request speed is now only limited by the the memory controller (which already goes at TB/sec in x86 hardware), a fast storage controller now has the opportunity to optimize and batch requests downstream to storage devices much much more efficiently. Because if the storage devices go at a certain speed but suddenly the addressing infrastructure is A LOT LOT faster, your optimization window just went through the roof and you can coordinate much more effectively. I forget the exact architecture, but one of IBM's 128-bit boxen already does this. Various random bits of the hardware use MMIO as a first-class addressing strategy. The OS does the rough equivalent of `disk = mmap2(/dev/sda)` at bootup. Maybe this is a z series thing.
  - mannschott 7y ago
    https://en.wikipedia.org/wiki/IBM_System_i https://en.wikipedia.org/wiki/IBM_System_i https://en.wikipedia.org/wiki/Single-level_store https://en.wikipedia.org/wiki/Single-level_store
  - anticensor 7y ago
    What about DMA over IPv6? Using a single 256-bit address, you would be able to address any byte in any IPv6-enabled computer directly.
  - hvidgaard 7y ago
    The calculations are off. 128 bit addressing allow for more than 3.40 x 10^38 addresses, and your storage example is 1.76 x 10^18 bits. That is, if we address the individual bits (we do not), we don't even have a name for the unit denoting that magnitude of space addressable left when using 128 bit addresses.
- usr1106 7y ago
  > Why would we want to switch to 128 bit addresses Predicting the future is difficult. 30 years ago (I am old enough to remember) it was hard to imagine that every household would have 10s of devices connected to the internet. My university VAX for 16 concurrent student users had less memory than required to show the splash screen when today's phone boots. So if in 30 years the computing paradigm has changed and we directly address memory over the internet? I must admit that in my imagination we have reached a point where growth will slow down. But I have learned that my imagination is not always good enough.
  - hvidgaard 7y ago
    Direct addressing that way seems unfeasible, it would require a extending the IP stack to support direct addressing for it to work, and it already support 128 bit addresses. A far more feasible model is direct addressing through a unique IP, and we do not need a bigger address space for that. That said, considering the current amounts of data Google holds, I could see the theoretical point about unique addressing every single byte they have. 64bit addressing only allow for a single order of magnitude of growth in that scenario.
- trasz 7y ago
  Thing is, an “address” could be much more than an address. One example is the CHERI hardware capability stuff and its Arm counterpart, Morello: sizeof(void *) is 16 there. And yeah, getting real world software, such as Postgres or Nginx, to work requires some fixes, but it’s really not that bad.

tedunangst 7y ago

Nobody ever thought that, and mindlessly repeating just means we don't learn anything.

eru 7y ago

Why would we want to switch to 128 bit addresses? Do you know how much memory we can address with 64 bit? (64 bit for addresses was arguably a mistake. 48 bit word size would probably have been better, but doesn't sound as cool.) Having said that, 128 bit can make sense for certain calculations. So floating point calculations and GPUs support long registers for some of what they are doing. (And having said that, Google figured out that they don't actually need all that precision when using GPUs for machine learning, and made TPUs with much smaller words.)

imron 7y ago
> Why would we want to switch to 128 bit addresses? How else are you going to have more than 16 exabytes of RAM?
- eru 7y ago
  Exactly, you probably won't. And I don't mean that in an absolute sense. People might very well get up to those orders of magnitude of RAM; but you are unlikely to have that amount of RAM available to a single processor. The extra margin of the 64 bit might help with address space randomization, though.
- jagger27 7y ago
  Memory banking is the obvious and ancient solution. PAE is another.
  - amluto 7y ago
    PAE is a kludge because the architects needed to add another tiny page table level to keep 32 bits of virtual space, but didn’t really want to add another level. The concept of having more physical address bits than virtual bits is reasonable, although it falls apart a bit with virtualization. The idea of having magic registers that fill themselves in for you and can’t be read at all (such as the PAE PDPTR registers) is a bad idea that unfortunately repeats itself in x86 design. Architects: don’t do this.
jnwatson 7y ago
Your modern Intel chip only has 48 addressable virtual address bits. The recent Ice Lake processors support 57 bits. I deal daily with Python processes that have "6.1t" in the "resident" column in top. Hitting the 48-bit 256 TiB limit isn't science fiction to me. I can already hit that with a moderate amount of effort (by memory mapping ~100 large kv-stores). It isn't that I need to do it, but I could imagine that someone does. It is well and proper that Intel rounded up to 64 bits. It should serve us well for the next 40ish years, which is good enough for my professional lifetime at least.
- eru 7y ago
  Yes, that's partially why I mentioned the 48 bits. It's interesting that you mention hitting that with memory mapping in practice. That is a valid concern.
dexen 7y ago
"Why would we want to switch to 128 bit addresses?" - eru, 2019 I am gonna hold onto your quote, for enjoyment and giggles in 2032 :^)
- dependenttypes 7y ago
  This is only 13 years from now. I really doubt that people will need 16 EiB by then in a single processor for a single process. Plus by that point you will probably have moved to message passing between distributed systems or something.
  - eru 7y ago
    My prediction is what we'll mostly see more of the same, with existing trends continuing. Ie more mobile, and more parallelism on the server side. But yeah, no 16 EiB on a single processor. (Though we might see people memory map crazy amounts, without ever actually accessing all of them, of course.)
exikyut 7y ago
The fact that you can put everything, ever into 128-bit means you can easily shove ten thousand 20-terabyte storage devices into a single 128-bit system's memory-mapped I/O space and still have 3 exabytes left over. No more packet-switched serial storage I/O. You now have first-class ability to ask for any byte anywhere, really really really fast. Because I/O request speed is now only limited by the the memory controller (which already goes at TB/sec in x86 hardware), a fast storage controller now has the opportunity to optimize and batch requests downstream to storage devices much much more efficiently. Because if the storage devices go at a certain speed but suddenly the addressing infrastructure is A LOT LOT faster, your optimization window just went through the roof and you can coordinate much more effectively. I forget the exact architecture, but one of IBM's 128-bit boxen already does this. Various random bits of the hardware use MMIO as a first-class addressing strategy. The OS does the rough equivalent of `disk = mmap2(/dev/sda)` at bootup. Maybe this is a z series thing.
- mannschott 7y ago
  https://en.wikipedia.org/wiki/IBM_System_i https://en.wikipedia.org/wiki/IBM_System_i https://en.wikipedia.org/wiki/Single-level_store https://en.wikipedia.org/wiki/Single-level_store
- anticensor 7y ago
  What about DMA over IPv6? Using a single 256-bit address, you would be able to address any byte in any IPv6-enabled computer directly.
- hvidgaard 7y ago
  The calculations are off. 128 bit addressing allow for more than 3.40 x 10^38 addresses, and your storage example is 1.76 x 10^18 bits. That is, if we address the individual bits (we do not), we don't even have a name for the unit denoting that magnitude of space addressable left when using 128 bit addresses.
usr1106 7y ago
> Why would we want to switch to 128 bit addresses Predicting the future is difficult. 30 years ago (I am old enough to remember) it was hard to imagine that every household would have 10s of devices connected to the internet. My university VAX for 16 concurrent student users had less memory than required to show the splash screen when today's phone boots. So if in 30 years the computing paradigm has changed and we directly address memory over the internet? I must admit that in my imagination we have reached a point where growth will slow down. But I have learned that my imagination is not always good enough.
- hvidgaard 7y ago
  Direct addressing that way seems unfeasible, it would require a extending the IP stack to support direct addressing for it to work, and it already support 128 bit addresses. A far more feasible model is direct addressing through a unique IP, and we do not need a bigger address space for that. That said, considering the current amounts of data Google holds, I could see the theoretical point about unique addressing every single byte they have. 64bit addressing only allow for a single order of magnitude of growth in that scenario.
trasz 7y ago
Thing is, an “address” could be much more than an address. One example is the CHERI hardware capability stuff and its Arm counterpart, Morello: sizeof(void *) is 16 there. And yeah, getting real world software, such as Postgres or Nginx, to work requires some fixes, but it’s really not that bad.