3 ms·

It would be cool to have a filesystem that unique'd storage of identical objects under the hood. E.g. even though the user sees a particular filename, it's actu

It would be cool to have a filesystem that unique'd storage of identical objects under the hood. E.g. even though the user sees a particular filename, it's actual stored using a SHA to detect duplicates. If you edit a file, it could do a copy-on-write, to ensure you're not inadvertently changing a different file.

db48x 8y ago
Deduplicating filesystems do exist; ZFS is one of the better examples. Of course, maintaining the indexes turns out to become a larger percentage of your cpu time, and those indexes have to live somewhere so they occupy a lot of memory. (You could reduce that impact by doing file-level rather than block -level deduplication, but then you also lose many of the benefits) If you want something that's higher level, there are tools like git annex that do this.
- ComputerGuru 8y ago
  ZFS (and others) feature block-based deduplication, not rule-level deduplication which is cheap and fast (though less “effective”).
  - ComputerGuru 8y ago
    *file-level
rdc12 8y ago
There was Venti in Plan9 that had this style of approach. [1] https://www.usenix.org/legacy/events/fast02/quinlan/quinlan_html/ https://www.usenix.org/legacy/events/fast02/quinlan/quinlan_...