4 ms·

Indeed. "Cloud" is essentially tons of commodity hardware nodes running a tailored userspace distributed file system application. There now certainly exist clou

Indeed. "Cloud" is essentially tons of commodity hardware nodes running a tailored userspace distributed file system application. There now certainly exist cloud services that host well in excess of 2^64 bytes, but they are not hosted on traditional file systems, but an array of those. I can't certainly blame their choice though, and 16-million terabyte hard drive arrays (the limit of 64-bit addressing) on a large mainframe are really just a few quantum leaps away. Heck, I still vividly recall buying a MASSIVE 40 GB HDD in 2000 - it seemed excessive beyond wildest dreams at the time (and was promptly filled to brim at a LAN party.) Since, advances in mass storage tech, most importantly the successful commercialization of perpendicular recording in 2005, mean that nowadays 12 TB drives are commercially viable - roughly 1000x the size of a 40 GB drive. In the next 20 years, petabyte-sized drives might well be available (though almost certainly not based on any form of magnetic recording). Once you hook hundred 50-petabyte drives to a single fileserver, you start closing in on limitations of 64-bit addressing.

killjoywashere 9y ago
Isn't there an XKCD about the limit of flash storage being 3 PB per gallon?
- mewse 9y ago
  I think you may be referring to an image from this What-If article, where the capacity of a gallon of MicroSD cards is estimated at 1.6PB per gallon: https://what-if.xkcd.com/31/ https://what-if.xkcd.com/31/
  - dimensi0nal 9y ago
    It's 12.8PB per gallon now.
    - perl4ever 9y ago
      A gallon of flash cards is great, but you do have to figure out which one you want after they are delivered. I'm wondering whether there is such a thing as an automated flash card library. I know people have archives of tapes like that.
      - Dylan16807 9y ago
        The thing about flash is that it's cheap and easy to keep it all permanently online. You can get a stack of 2.5 inch SSDs for the same price as microsd cards, and plugging them into a bus takes less space than a tape robot. When you're looking at the same price for two systems, and one is 100x-1000x faster at the cost of being 'merely' hundreds of TB per gallon instead of a few PB per gallon, nobody is going to bother engineering the latter just to save a few bucks on shipping.
      - detaro 9y ago
        Tape is a lot better at keeping data while in storage though.
dsr_ 9y ago
Overly pedantic quibble: a quantum of anything is the minimum possible unit. One planck length, time, mass, charge, temperature. Everybody uses it to mean "big". Oh, language.
- dmix 9y ago
  But is a "Quantum leap" really equivalent to leaping one planck? Honest question. I assumed it implied something far 'larger' such as a leap through spacetime...
  - aidenn0 9y ago
    I believe the term originally applied to electron energy level transitions within an atom, which only vaguely correlates with anything one might call distance. The interesting thing there was that the electrons existed only at discrete energy levels, and the transition between them appeared instantaneous.
  - ianai 9y ago
    I always assumed that sprang from the tv show with that name.
  - white-flame 9y ago
    A quantum leap is a discontinuity, a change that cannot happen gradually. It typically implies a revolutionary change over evolutionary improvement.
- taneq 9y ago
  "Quantum leap" is typically interpreted as "big leap" but I believe the true implication is more like "indivisible leap", in line with your quibble. A Turing-complete computer is a "quantum leap" ahead of a non-Turing-complete calculator. A warp drive is a "quantum leap" ahead of a reaction drive. You either have all of it or none of it.
  - quickthrower2 9y ago
    I know who we should ask: Ziggy
versteegen 9y ago
The usage in the article is that 64/128 bits is the addressing used for the blocks. A block is 512 bytes. So a 64-bit filesystem would actually be limited to 2^73 bytes = 8192 EiB = 8.6 billion TiB of storage. No cloud vendor has a billion harddisks, that's for sure. (The filesystem might only support a maximum file size of 2^64 bytes, but that's nothing to do with the capacity of the storage array)
- wyoung2 9y ago
  The article isn't talking about the size of raw disk blocks, better called the sector size. It's talking about ZFS blocks, which are variable in size, defaulting to 128 kB: https://en.wikipedia.org/wiki/ZFS#Variable_block_sizes https://en.wikipedia.org/wiki/ZFS#Variable_block_sizes ZFS aside, the 512 byte sector isn't even a hard limit at the storage medium level. Disks with 528 byte sectors — and other odd sizes — have been available in the enterprise world for a long time now: https://en.wikipedia.org/wiki/Hard_disk_drive#Market_segments https://en.wikipedia.org/wiki/Hard_disk_drive#Market_segment... HDDs and SSDs with 4 kB sectors dominate the consumer market now: https://en.wikipedia.org/wiki/Advanced_Format https://en.wikipedia.org/wiki/Advanced_Format

killjoywashere 9y ago

Isn't there an XKCD about the limit of flash storage being 3 PB per gallon?

mewse 9y ago
I think you may be referring to an image from this What-If article, where the capacity of a gallon of MicroSD cards is estimated at 1.6PB per gallon: https://what-if.xkcd.com/31/ https://what-if.xkcd.com/31/
- dimensi0nal 9y ago
  It's 12.8PB per gallon now.
  - perl4ever 9y ago
    A gallon of flash cards is great, but you do have to figure out which one you want after they are delivered. I'm wondering whether there is such a thing as an automated flash card library. I know people have archives of tapes like that.
    - Dylan16807 9y ago
      The thing about flash is that it's cheap and easy to keep it all permanently online. You can get a stack of 2.5 inch SSDs for the same price as microsd cards, and plugging them into a bus takes less space than a tape robot. When you're looking at the same price for two systems, and one is 100x-1000x faster at the cost of being 'merely' hundreds of TB per gallon instead of a few PB per gallon, nobody is going to bother engineering the latter just to save a few bucks on shipping.
    - detaro 9y ago
      Tape is a lot better at keeping data while in storage though.

dsr_ 9y ago

Overly pedantic quibble: a quantum of anything is the minimum possible unit. One planck length, time, mass, charge, temperature. Everybody uses it to mean "big". Oh, language.

dmix 9y ago
But is a "Quantum leap" really equivalent to leaping one planck? Honest question. I assumed it implied something far 'larger' such as a leap through spacetime...
- aidenn0 9y ago
  I believe the term originally applied to electron energy level transitions within an atom, which only vaguely correlates with anything one might call distance. The interesting thing there was that the electrons existed only at discrete energy levels, and the transition between them appeared instantaneous.
- ianai 9y ago
  I always assumed that sprang from the tv show with that name.
- white-flame 9y ago
  A quantum leap is a discontinuity, a change that cannot happen gradually. It typically implies a revolutionary change over evolutionary improvement.
taneq 9y ago
"Quantum leap" is typically interpreted as "big leap" but I believe the true implication is more like "indivisible leap", in line with your quibble. A Turing-complete computer is a "quantum leap" ahead of a non-Turing-complete calculator. A warp drive is a "quantum leap" ahead of a reaction drive. You either have all of it or none of it.
- quickthrower2 9y ago
  I know who we should ask: Ziggy

versteegen 9y ago

The usage in the article is that 64/128 bits is the addressing used for the blocks. A block is 512 bytes. So a 64-bit filesystem would actually be limited to 2^73 bytes = 8192 EiB = 8.6 billion TiB of storage. No cloud vendor has a billion harddisks, that's for sure. (The filesystem might only support a maximum file size of 2^64 bytes, but that's nothing to do with the capacity of the storage array)

wyoung2 9y ago
The article isn't talking about the size of raw disk blocks, better called the sector size. It's talking about ZFS blocks, which are variable in size, defaulting to 128 kB: https://en.wikipedia.org/wiki/ZFS#Variable_block_sizes https://en.wikipedia.org/wiki/ZFS#Variable_block_sizes ZFS aside, the 512 byte sector isn't even a hard limit at the storage medium level. Disks with 528 byte sectors — and other odd sizes — have been available in the enterprise world for a long time now: https://en.wikipedia.org/wiki/Hard_disk_drive#Market_segments https://en.wikipedia.org/wiki/Hard_disk_drive#Market_segment... HDDs and SSDs with 4 kB sectors dominate the consumer market now: https://en.wikipedia.org/wiki/Advanced_Format https://en.wikipedia.org/wiki/Advanced_Format