4 ms·

I'd be curious if anyone with understanding of memory transistor density has an idea of what sort of physical footprint and power requirements 128-bit addressab

I'd be curious if anyone with understanding of memory transistor density has an idea of what sort of physical footprint and power requirements 128-bit addressable memory would look like, because I'm having a hard time picturing even a distributed, shared virtual memory space realistically exceeding 16 exabytes. It looks like the Fugaku ARM cluster with 158,000 nodes uses < 5 PiB for 40 MW. As an aside, man does reading something from @sgi.com on USENET take me back to a better time. Title should probably note [1995] [0] https://www.fujitsu.com/global/about/innovation/fugaku/specifications/ https://www.fujitsu.com/global/about/innovation/fugaku/speci... [1] Also https://arxiv.org/abs/quant-ph/9908043 https://arxiv.org/abs/quant-ph/9908043 "Ultimate physical limits to computation"

Laremere 5y ago
85.8 grams of DNA is enough to overflow 64bits. Not that we're anywhere close to being able to use or process information on those scales, but it at least storage on that scale doesn't outright violate the laws of physics/information theory.
- littlestymaar 5y ago
  Storage needs don't affect pointer size though. To need bigger pointers, you must need an addressable space this big. That said, how to you make your calculation about DNA's information density? Without counting water, I've arrived at 720g for one “mole of bits”, which is a bunch of orders of magnitude higher than your figure: 2 amino-acids (262g/mole) [1], 2 Deoxyribose (135g/mole each) and 2 Phosphate (94g/mole each). [1]: and it's the same value be it A-T or G-C, that's interesting
  - spqr0a1 5y ago
    The information density is even higher than that (double), because one pair of bases has 4 possible states. The pair A-T is symbolically distinct from T-A, for example.
- jamal-kumar 5y ago
  Not anywhere close? I've heard things about using DNA itself for the memory storage format which is super interesting. I guess it just has bandwidth limitations and is more for long term storage? [1] [1] https://www.scientificamerican.com/article/dna-the-ultimate-data-storage-solution/ https://www.scientificamerican.com/article/dna-the-ultimate-...
  - d_tr 5y ago
    85 grams of DNA contain around 8 * 10^22 base pairs, 2 bits per pair, so around 2 * 10^22 bytes. So you need about 75 bits to address all these bytes. I do not know why Laremere chose 85.8 grams though as these are quite above the 64 bit limit, unless I got my multiplications wrong :P
    - sdenton4 5y ago
      Not that anything uses that much DNA, anyway: "The male nuclear diploid genome extends for 6.27 Gigabase pairs (Gbp), is 205.00 cm (cm) long and weighs 6.41 picograms (pg)." Or, many many orders of magnitude less than 85 grams...
jandrewrogers 5y ago
It is not too difficult to imagine for virtual memory. Servers with high storage density are right up against the current virtual memory limits such that you can't mmap() storage as it is. Not that you'd want to mmap() if you cared about performance. Similarly, the largest data models have been in the exabyte range for a while. It is just a matter of time before they are not byte-addressable using a 64-bit integer. I think there is more utility in 128-bit ALUs than 128-bit pointers, as a practical matter.
- zepearl 5y ago
  > Not that you'd want to mmap() if you cared about performance. So far I never used mmap (no usecase that pushed me to find alternatives for the usual open&seek), but I was always curious about it. Your statement made me search & find this ( https://unix.stackexchange.com/questions/474926/how-does-memory-mapping-a-file-have-significant-performance-increases-over-the-s https://unix.stackexchange.com/questions/474926/how-does-mem... ) => interesting. Thanks :)
  - BeeOnRope 5y ago
    You might also consider this Q&A whose answers contain several reasons mmap might be faster, and several why it may be slower: https://stackoverflow.com/questions/45972/mmap-vs-reading-blocks https://stackoverflow.com/questions/45972/mmap-vs-reading-bl...
  - IgorPartola 5y ago
    mmap has a bunch of other use cases other than opening files. Shared memory, fast memory allocation, disk-backed memory regions. I believe it was invented after the gun that shoots both forwards and backwards failed to deliver on its fatality quota.
- phoe-krk 5y ago
  A machine word/pointer as wide as 72 bits would already solve a lot of problems that appear in e.g. dynamic languages nowadays, such as an inability to represent an IEEE double float (64 bits wide) as a value without any sort of type tagging. Still, I don't think anyone nowadays would settle for a non-power-of-2 machine word size, even if just for backwards compatibility and even if a pointer is limited to e.g. 72 bits out of all 128. If that is the case, then another use could be a "fat pointer" that was proposed somewhere in the C world to avoid passing raw pointers without their lengths.You could pass a length of an array along with its address as long as the length of your array can fit in e.g. 56 bits, which would be enough for most use cases if we feel like going into the "640kB is enough for everyone" route again.
jamal-kumar 5y ago
4.85 PiB of RAM -- WOW
- jasonwatkinspdx 5y ago
  Oh that machine is a monster. All the memory is HBM2 as well. So it's stacked in the same package as each processor chip, connected with an interposer. HBM gives you 4 stacks, each with 8 single duplex 128 bit memory buses, that can run at 2+ ghz. So that 4.85 PiB is accessible with a raw bandwidth of around 144 PiB/sec. Definitely a monster of a machine.
- hajile 5y ago
  They are at the edge of current RAM limits of 52-bits (I believe this is the case on both x86 and ARM systems). If they could get a system with the full 64-bits of addressable RAM, they'd likely have gone even bigger.
pinewurst 5y ago
It’s incorrect to view Fugaku (as cool as it is) as having a monolithically addressable 5 PiB. Each processor/node has 32 GiB directly attached via HBM and not expandable (Apple M1ish). It’s a really big cluster of (much) smaller memory spaces.
vimax 5y ago
From Jeff Bonwick, ZFS Developer: Although we'd all like Moore's Law to continue forever, quantum mechanics imposes some fundamental limits on the computation rate and information capacity of any physical device. In particular, it has been shown that 1 kilogram of matter confined to 1 liter of space can perform at most 10^51 operations per second on at most 1031 bits of information [see Seth Lloyd, "Ultimate physical limits to computation." Nature 406, 1047-1054 (2000)]. A fully-populated 128-bit storage pool would contain 2^128 blocks = 2^137 bytes = 2140 bits; therefore the minimum mass required to hold the bits would be (2^140 bits) / (10^31 bits/kg) = 136 billion kg. That's a lot of gear. To operate at the 10^31 bits/kg limit, however, the entire mass of the computer must be in the form of pure energy. By E=mc^2, the rest energy of 136 billion kg is 1.2x10^28 J. The mass of the oceans is about 1.4x10^21 kg. It takes about 4,000 J to raise the temperature of 1 kg of water by 1 degree Celcius, and thus about 400,000 J to heat 1 kg of water from freezing to boiling. The latent heat of vaporization adds another 2 million J/kg. Thus the energy required to boil the oceans is about 2.4x10^6 J/kg * 1.4x10^21 kg = 3.4x1^027 J. Thus, fully populating a 128-bit storage pool would, literally, require more energy than boiling the oceans.

Laremere 5y ago

85.8 grams of DNA is enough to overflow 64bits. Not that we're anywhere close to being able to use or process information on those scales, but it at least storage on that scale doesn't outright violate the laws of physics/information theory.

littlestymaar 5y ago
Storage needs don't affect pointer size though. To need bigger pointers, you must need an addressable space this big. That said, how to you make your calculation about DNA's information density? Without counting water, I've arrived at 720g for one “mole of bits”, which is a bunch of orders of magnitude higher than your figure: 2 amino-acids (262g/mole) [1], 2 Deoxyribose (135g/mole each) and 2 Phosphate (94g/mole each). [1]: and it's the same value be it A-T or G-C, that's interesting
- spqr0a1 5y ago
  The information density is even higher than that (double), because one pair of bases has 4 possible states. The pair A-T is symbolically distinct from T-A, for example.
jamal-kumar 5y ago
Not anywhere close? I've heard things about using DNA itself for the memory storage format which is super interesting. I guess it just has bandwidth limitations and is more for long term storage? [1] [1] https://www.scientificamerican.com/article/dna-the-ultimate-data-storage-solution/ https://www.scientificamerican.com/article/dna-the-ultimate-...
- d_tr 5y ago
  85 grams of DNA contain around 8 * 10^22 base pairs, 2 bits per pair, so around 2 * 10^22 bytes. So you need about 75 bits to address all these bytes. I do not know why Laremere chose 85.8 grams though as these are quite above the 64 bit limit, unless I got my multiplications wrong :P
  - sdenton4 5y ago
    Not that anything uses that much DNA, anyway: "The male nuclear diploid genome extends for 6.27 Gigabase pairs (Gbp), is 205.00 cm (cm) long and weighs 6.41 picograms (pg)." Or, many many orders of magnitude less than 85 grams...

jandrewrogers 5y ago

It is not too difficult to imagine for virtual memory. Servers with high storage density are right up against the current virtual memory limits such that you can't mmap() storage as it is. Not that you'd want to mmap() if you cared about performance. Similarly, the largest data models have been in the exabyte range for a while. It is just a matter of time before they are not byte-addressable using a 64-bit integer. I think there is more utility in 128-bit ALUs than 128-bit pointers, as a practical matter.

zepearl 5y ago
> Not that you'd want to mmap() if you cared about performance. So far I never used mmap (no usecase that pushed me to find alternatives for the usual open&seek), but I was always curious about it. Your statement made me search & find this ( https://unix.stackexchange.com/questions/474926/how-does-memory-mapping-a-file-have-significant-performance-increases-over-the-s https://unix.stackexchange.com/questions/474926/how-does-mem... ) => interesting. Thanks :)
- BeeOnRope 5y ago
  You might also consider this Q&A whose answers contain several reasons mmap might be faster, and several why it may be slower: https://stackoverflow.com/questions/45972/mmap-vs-reading-blocks https://stackoverflow.com/questions/45972/mmap-vs-reading-bl...
- IgorPartola 5y ago
  mmap has a bunch of other use cases other than opening files. Shared memory, fast memory allocation, disk-backed memory regions. I believe it was invented after the gun that shoots both forwards and backwards failed to deliver on its fatality quota.
phoe-krk 5y ago
A machine word/pointer as wide as 72 bits would already solve a lot of problems that appear in e.g. dynamic languages nowadays, such as an inability to represent an IEEE double float (64 bits wide) as a value without any sort of type tagging. Still, I don't think anyone nowadays would settle for a non-power-of-2 machine word size, even if just for backwards compatibility and even if a pointer is limited to e.g. 72 bits out of all 128. If that is the case, then another use could be a "fat pointer" that was proposed somewhere in the C world to avoid passing raw pointers without their lengths.You could pass a length of an array along with its address as long as the length of your array can fit in e.g. 56 bits, which would be enough for most use cases if we feel like going into the "640kB is enough for everyone" route again.

jamal-kumar 5y ago

4.85 PiB of RAM -- WOW

jasonwatkinspdx 5y ago
Oh that machine is a monster. All the memory is HBM2 as well. So it's stacked in the same package as each processor chip, connected with an interposer. HBM gives you 4 stacks, each with 8 single duplex 128 bit memory buses, that can run at 2+ ghz. So that 4.85 PiB is accessible with a raw bandwidth of around 144 PiB/sec. Definitely a monster of a machine.
hajile 5y ago
They are at the edge of current RAM limits of 52-bits (I believe this is the case on both x86 and ARM systems). If they could get a system with the full 64-bits of addressable RAM, they'd likely have gone even bigger.

pinewurst 5y ago

It’s incorrect to view Fugaku (as cool as it is) as having a monolithically addressable 5 PiB. Each processor/node has 32 GiB directly attached via HBM and not expandable (Apple M1ish). It’s a really big cluster of (much) smaller memory spaces.

vimax 5y ago

From Jeff Bonwick, ZFS Developer: Although we'd all like Moore's Law to continue forever, quantum mechanics imposes some fundamental limits on the computation rate and information capacity of any physical device. In particular, it has been shown that 1 kilogram of matter confined to 1 liter of space can perform at most 10^51 operations per second on at most 1031 bits of information [see Seth Lloyd, "Ultimate physical limits to computation." Nature 406, 1047-1054 (2000)]. A fully-populated 128-bit storage pool would contain 2^128 blocks = 2^137 bytes = 2140 bits; therefore the minimum mass required to hold the bits would be (2^140 bits) / (10^31 bits/kg) = 136 billion kg. That's a lot of gear. To operate at the 10^31 bits/kg limit, however, the entire mass of the computer must be in the form of pure energy. By E=mc^2, the rest energy of 136 billion kg is 1.2x10^28 J. The mass of the oceans is about 1.4x10^21 kg. It takes about 4,000 J to raise the temperature of 1 kg of water by 1 degree Celcius, and thus about 400,000 J to heat 1 kg of water from freezing to boiling. The latent heat of vaporization adds another 2 million J/kg. Thus the energy required to boil the oceans is about 2.4x10^6 J/kg * 1.4x10^21 kg = 3.4x1^027 J. Thus, fully populating a 128-bit storage pool would, literally, require more energy than boiling the oceans.