3 ms·

From reading the first paragraph it sounds like the author will explain to great lengths why Dropbox had it hard to support filesystems other than ext4. But th

by bjhkx 7y ago

From reading the first paragraph it sounds like the author will explain to great lengths why Dropbox had it hard to support filesystems other than ext4. But then you read the article and it goes to explain how dealing with files is hard and sometimes data corruption and loss occurs (fair enough), but nothing filesystem specific that could explain why ext4 is superior (or special) and had to be chosen. So it reads kind of like an excuse for Dropbox, only that it isn't.

ChrisSD 7y ago
ext4 is by far the most popular filesystem so supporting it is a necessity for that reason alone. Nothing to do with it being superior or special. The argument this article is making is that supporting additional filesystems is hard. The is meant to refute the allegation that it's trivial to add whatever other filesystems the OS supports.
- bjhkx 7y ago
  No, that's the argument this article is supposedly making. But it's not actually making it. That's the problem. Filesystems are an abstraction and I expected to see some problems regarding a leaky abstraction or something, but the article doesn't mention anything like that. It mentions that dealing with hard disks is hard (which I believe, since they are flaky hardware). But dropbox didn't say "we won't be supporting this kind of hard disk/hard disk controller", but "we won't be supporting these filesystems". Where's the proof that those filesystems have problems that ext4 doesn't have?
  - yorwba 7y ago
    From the article: "Large parts of the file API look like this, where behavior varies across filesystems or across different modes of the same filesystem. For example, if we look at mainstream filesystems, appends are atomic, except when using ext3 or ext4 with data=writeback, or ext2 in any mode and directory operations can't be re-ordered w.r.t. any other operations, except on btrfs." That doesn't mean ext4 is problem-free, but it does mean that other filesystems have different problems that are not fixed by mitigations for ext4's quirks.
    - bjhkx 7y ago
      Okay and? Why does userspace care about this? Should Emacs not run on btrfs?
      - pjc50 7y ago
        This is rather like the bluescreen problem: if your application tries to open a file from the normal filesystem and it's corrupt, the user blames the application. If the user opens a file in the Dropbox folder, they blame Dropbox. So they end up engaging in heroics to not be blamed for it. (Windows has gone to increasing lengths to accomodate and contain badly written drivers, since most bluescreens are caused by drivers. There is now a subsystem to allow the video drivers to crash and entirely restart without bluescreening.)
        namibj 7y ago
        Which prevents any CUDA kernels from running longer than like 5 seconds or so. Which means you can't use the GPU to spawn it's own kernels with no PCIe/driver latency in between, because this master has to finish before windows kills it. Last I checked, providing callback function pointers to binary vendor libraries (read/write adapters for FFT come to mind, allowing on-the-fly metric computation or skipping an intermediate storage for FFT convolution) was only possible on Linux, and with statically linking said vendor library into the software (incidentally breaking binary distributability for GPL).

by bjhkx 7y ago

ChrisSD 7y ago

ext4 is by far the most popular filesystem so supporting it is a necessity for that reason alone. Nothing to do with it being superior or special. The argument this article is making is that supporting additional filesystems is hard. The is meant to refute the allegation that it's trivial to add whatever other filesystems the OS supports.

bjhkx 7y ago
No, that's the argument this article is supposedly making. But it's not actually making it. That's the problem. Filesystems are an abstraction and I expected to see some problems regarding a leaky abstraction or something, but the article doesn't mention anything like that. It mentions that dealing with hard disks is hard (which I believe, since they are flaky hardware). But dropbox didn't say "we won't be supporting this kind of hard disk/hard disk controller", but "we won't be supporting these filesystems". Where's the proof that those filesystems have problems that ext4 doesn't have?
- yorwba 7y ago
  From the article: "Large parts of the file API look like this, where behavior varies across filesystems or across different modes of the same filesystem. For example, if we look at mainstream filesystems, appends are atomic, except when using ext3 or ext4 with data=writeback, or ext2 in any mode and directory operations can't be re-ordered w.r.t. any other operations, except on btrfs." That doesn't mean ext4 is problem-free, but it does mean that other filesystems have different problems that are not fixed by mitigations for ext4's quirks.
  - bjhkx 7y ago
    Okay and? Why does userspace care about this? Should Emacs not run on btrfs?
    - pjc50 7y ago
      This is rather like the bluescreen problem: if your application tries to open a file from the normal filesystem and it's corrupt, the user blames the application. If the user opens a file in the Dropbox folder, they blame Dropbox. So they end up engaging in heroics to not be blamed for it. (Windows has gone to increasing lengths to accomodate and contain badly written drivers, since most bluescreens are caused by drivers. There is now a subsystem to allow the video drivers to crash and entirely restart without bluescreening.)
      - namibj 7y ago
        Which prevents any CUDA kernels from running longer than like 5 seconds or so. Which means you can't use the GPU to spawn it's own kernels with no PCIe/driver latency in between, because this master has to finish before windows kills it. Last I checked, providing callback function pointers to binary vendor libraries (read/write adapters for FFT come to mind, allowing on-the-fly metric computation or skipping an intermediate storage for FFT convolution) was only possible on Linux, and with statically linking said vendor library into the software (incidentally breaking binary distributability for GPL).