4 ms·

No one deletes data: HDDs are useful for tiered storage and bulk bits. A number of file systems also support transparent tape usage: a stub is left in the dire

No one deletes data: HDDs are useful for tiered storage and bulk bits. A number of file systems also support transparent tape usage: a stub is left in the directory structure and if anyone tries reading it the bits are fetched from a robot.

catiopatio 3y ago
What file systems support transparent tape loading? That’s a neat feature, but I’d expect a lot of software to behave badly if local file system operations suddenly started exhibiting tape-level latency.
- derefr 3y ago
  Windows Storage Spaces. IIRC, a Tiered Storage Space means that evicted-to-tape files that get OpenFile'd, get copied in full back to the hot disk before the OpenFile completes, such that any IO then proceeds from the hot disk, not from tape.
- AndrewDucker 3y ago
  IBM Z/OS supports it. You'd try to open a file and it would pop up a "retrieving" message and you'd know you could go and get a cup of coffee while you waited for your file. (That's obviously at the OS layer rather than the filesystem layer. But the application didn't know where the file was until it asked for it.)
- throw0101c 3y ago
  > What file systems support transparent tape loading? In the enterprise/HPC space, GPFS and Lustre.
- projektfu 3y ago
  NTFS, if I remember correctly.
hinkley 3y ago
It would be nice if more communications protocols achieved feature parity with Ethernet. Or even USB. Give me a hard drive with a connector I can plug into a hub or switch and just keep daisy chaining the oldest hardware deeper and deeper, instead of needing a robot to physically move them around. It’s not like adding a couple microseconds to response time will even be noticed on a crappy old hard drive. Add a deep sleep mode and bam, you got a stew goin’.
- convolvatron 3y ago
  As I'm sure you're aware, there was such a thing. But market segmentation was used to turn it into a higher super-enterprise tier. Unless you care a lot about latency variance, there is no reason why those two sets of protocols (networking and storage) shouldn't have converged a long time ago. Admittedly, 20 years ago there were some bandwidth issues due to run lengths, but those are long gone. The market is simultaneously the hand that giveth and taketh away
  - hinkley 3y ago
    Are you talking about PCIe fabrics? There was a moment there where I thought a shiny future was ahead of us and then… nothing. Still bitter about that. Or was this tried another way? I’m hoping that open firmwares create an environment where we can start treating the computer as a network. It’s a slim hope but not entirely insane.
    - convolvatron 3y ago
      I was taking about Fibre Channel...same drive with a different controller would cost 2x, and the switches and management software were just obscenely priced. there was also iscsi. I tried to use to it in a couple jobs and it was really poorly performing, and not very reliable. I _suspect_ that's just because it was never paid attention to properly? maybe because it was using TCP instead of a reliable but unordered protocol. thinking about it, I'm off on the bandwidth numbers. SSDs move out quite a bit...so maybe you wouldn't replace your primary OS drive. But for secondary and tertiary storage ethernet drivers would really be nice PCIe fabrics are still kicking around. I agree that's another enabling technology that just hasn't found a home. You would think with this USBc stuff that would suddenly be of great interest.

catiopatio 3y ago

What file systems support transparent tape loading? That’s a neat feature, but I’d expect a lot of software to behave badly if local file system operations suddenly started exhibiting tape-level latency.

derefr 3y ago
Windows Storage Spaces. IIRC, a Tiered Storage Space means that evicted-to-tape files that get OpenFile'd, get copied in full back to the hot disk before the OpenFile completes, such that any IO then proceeds from the hot disk, not from tape.
AndrewDucker 3y ago
IBM Z/OS supports it. You'd try to open a file and it would pop up a "retrieving" message and you'd know you could go and get a cup of coffee while you waited for your file. (That's obviously at the OS layer rather than the filesystem layer. But the application didn't know where the file was until it asked for it.)
throw0101c 3y ago
> What file systems support transparent tape loading? In the enterprise/HPC space, GPFS and Lustre.
projektfu 3y ago
NTFS, if I remember correctly.

hinkley 3y ago

It would be nice if more communications protocols achieved feature parity with Ethernet. Or even USB. Give me a hard drive with a connector I can plug into a hub or switch and just keep daisy chaining the oldest hardware deeper and deeper, instead of needing a robot to physically move them around. It’s not like adding a couple microseconds to response time will even be noticed on a crappy old hard drive. Add a deep sleep mode and bam, you got a stew goin’.

convolvatron 3y ago
As I'm sure you're aware, there was such a thing. But market segmentation was used to turn it into a higher super-enterprise tier. Unless you care a lot about latency variance, there is no reason why those two sets of protocols (networking and storage) shouldn't have converged a long time ago. Admittedly, 20 years ago there were some bandwidth issues due to run lengths, but those are long gone. The market is simultaneously the hand that giveth and taketh away
- hinkley 3y ago
  Are you talking about PCIe fabrics? There was a moment there where I thought a shiny future was ahead of us and then… nothing. Still bitter about that. Or was this tried another way? I’m hoping that open firmwares create an environment where we can start treating the computer as a network. It’s a slim hope but not entirely insane.
  - convolvatron 3y ago
    I was taking about Fibre Channel...same drive with a different controller would cost 2x, and the switches and management software were just obscenely priced. there was also iscsi. I tried to use to it in a couple jobs and it was really poorly performing, and not very reliable. I _suspect_ that's just because it was never paid attention to properly? maybe because it was using TCP instead of a reliable but unordered protocol. thinking about it, I'm off on the bandwidth numbers. SSDs move out quite a bit...so maybe you wouldn't replace your primary OS drive. But for secondary and tertiary storage ethernet drivers would really be nice PCIe fabrics are still kicking around. I agree that's another enabling technology that just hasn't found a home. You would think with this USBc stuff that would suddenly be of great interest.