5 ms·

Ask HN: I'd like an OS/FileSystem that Never Deletes Anything. Does it exist?

• alexvoda 5y ago
  Maybe one option would be to host a WebDav share with the Delta-V extension (RFC 3253) backed by Subversion. An alternative to Subversion might me NextCloud which supports automatic versioning and WebDav, but I do not know if it supports versioning through WebDav. I also desire an easy solution to this usecase.
  • alexvoda 5y ago
    To expand a bit on the issues of the solution above, the problem with WebDav/Delta-V is that very few servers and very few clients suport it and it is also dificult to find documentation on whehter it is or isn't supported by a piece of software. Here is more information about WebDav/Delta-V: https://www.linuxtopia.org/online_books/programming_tool_guides/version_control_with_subversion/svn.webdav.html https://www.linuxtopia.org/online_books/programming_tool_gui... This part describes what you and I desire with caveats: https://www.linuxtopia.org/online_books/programming_tool_guides/version_control_with_subversion/svn.webdav.autoversioning.html https://www.linuxtopia.org/online_books/programming_tool_gui... Here is a discussion on why IIS never implemented it: https://social.msdn.microsoft.com/Forums/en-US/a9bf95b1-36ac-4c51-9aee-bb26b87e0d9c/webdav-file-versions https://social.msdn.microsoft.com/Forums/en-US/a9bf95b1-36ac...
    • COMMENT___ 5y ago
      See also * https://svnbook.red-bean.com/nightly/en/svn.webdav.html https://svnbook.red-bean.com/nightly/en/svn.webdav.html * http://svn.apache.org/repos/asf/subversion/trunk/notes/http-and-webdav/ http://svn.apache.org/repos/asf/subversion/trunk/notes/http-...
• alexvoda 5y ago
  Just for reference, here are links to various old research papers referencing solutions to this scenario: A list of options from a report: https://www.fsl.cs.sunysb.edu/docs/secfs-rpe/index.html#tth_chAp4 https://www.fsl.cs.sunysb.edu/docs/secfs-rpe/index.html#tth_... VersionFS: https://www.fsl.cs.sunysb.edu/docs/versionfs-msthesis/index.html https://www.fsl.cs.sunysb.edu/docs/versionfs-msthesis/index.... Elephant: https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.70.8488 https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.70.... https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.129.7767 https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.129... Silver: https://www.usenix.org/system/files/conference/hotstorage16/hotstorage16_wei.pdf https://www.usenix.org/system/files/conference/hotstorage16/... Also possibly of interest, vfs: https://github.com/FooSoft/vfs https://github.com/FooSoft/vfs
• Someone 5y ago
  > Does a filesystem that never deletes/overwrites anything exist? > Obviously once space runs out it would start overwriting the oldest versions. You can’t have both. Also, your system wouldn’t prevent data loss on hardware failures. What you describe is/may be a WORM device (https://en.wikipedia.org/wiki/Write_once_read_many https://en.wikipedia.org/wiki/Write_once_read_many)
• Minor49er 5y ago
  I hope this hypothetical OS never needs to use a swap file. More seriously, in a time of unobtrusive automated backups and dirt cheap storage space, why is this such an issue where you want to have this kind of system?
  • hulitu 5y ago
    Domain OS had versioning filesystem. You could always load an older version of your file in the editor.
    • Minor49er 5y ago
      Something similar can be achieved today with a combination of autosaves or version saves at the application level as well as time machine backups at the OS level
• whalesalad 5y ago
  I imagine something does exist that does this, but I do not know what it is called. I know that overlayfs exists, which could enable this sort of thing by capturing each individual change to the filesystem as a new layer. https://www.kernel.org/doc/html/latest/filesystems/overlayfs.html?highlight=overlayfs https://www.kernel.org/doc/html/latest/filesystems/overlayfs... Btrfs and ZFS can do snapshotting, and I would imagine it could be configured to snapshot at certain intervals or after individual changes to the filesystem or a section of the filesystem, like watched directories. https://wiki.archlinux.org/title/Btrfs#Snapshots https://wiki.archlinux.org/title/Btrfs#Snapshots https://docs.oracle.com/cd/E19253-01/819-5461/gbcya/index.html https://docs.oracle.com/cd/E19253-01/819-5461/gbcya/index.ht... There's also filesystem-in-userland tech like Fuse which could enable something like this. https://www.kernel.org/doc/html/latest/filesystems/fuse.html https://www.kernel.org/doc/html/latest/filesystems/fuse.html
• Rebelgecko 5y ago
  Look up "append only" filesystems. That'll get you most of the way there, but you'll have to manually reset things to get rid of old files. Or just use a competent backup system.
  • choletentent 5y ago
    Let's say you are editing a source code file. You save and you run. Than you tweak, save and run again. But then you would like to go two changes in the past, but you did not make a commit. Very recent changes are not captured by backup systems, no matter how competent they are. I'd like to have all my files under git, for every save - and frictionless. I know it is a lot to ask.
    • typetheorist 5y ago
      This sounds a lot like what you're describing: https://github.com/tkellogg/dura https://github.com/tkellogg/dura
    • toast0 5y ago
      > Very recent changes are not captured by backup systems, no matter how competent they are. Depends, there's backup software for Windows that will watch for file changes and backup immediately. Usually enterprise targetted. You could certainly do that on other OSes, possibly with less nice apis; a half measure would be taking filesystem snapshots continuously or once a minute or ???. There's filesystems you can use on cd-r/dvd-r, although that doesn't sound pleasant, that would give you the option to go back to any version.
    • dagss 5y ago
      Look up "git wip", configure your editor to use it and it keeps an unintrusive 'second branch' in the background. https://atom.io/packages/git-wip https://atom.io/packages/git-wip "git-wip is a script that will manage Work In Progress (or WIP) branches. WIP branches are mostly throw away but identify points of development between commits. The intent is to tie this script into your editor so that each time you save your file, the git-wip script captures that state in git."
    • 5y ago
• LinuxBender 5y ago
  The expensive version of what you are describing is a vaulting appliance but I won't continue down that path unless you are a bank. The affordable version of what you are describing is a set of NAS class servers that use something like DRBD or Ceph for multi-node replication and rsnapshot for local snapshots/versioning if LVM. If using LVM, then rsnapshot should be configured to save its snapshot where the remote clients can not write to it. Look into setting up Ceph clusters if you plan to build something that will need to scale really large on individual volumes across multiple nodes. Ceph supports creating snapshots. There are pros and cons to solutions such as DRBD, LVM and Ceph. That would be a topic in and of itself. Others here are mentioning ZFS and that is also a popular solution but I have never used it and can't comment on it. In summary if I was asked to build something that would grow to unknown size, require snapshots to roll-back files to a specific date and be fault tolerant I would go with Ceph. As a bonus feature there are libraries and tools to present Ceph volumes as S3-like buckets. There are ansible playbooks for setting up Ceph clusters. Ceph has some really good security features as well.
• some_chap 5y ago
  Digital's VMS/OpenVMS had something that was a fair way towards this, in that a file was created with a version number (myfile.txt;1) and when modified, a 2nd file (myfile.txt;2) was created, and so on... In practice we used to work on things then purge when happy with the final changes, as we didn't have that much 'spare' disk, but it was a nice feature.
• __d 5y ago
  Check out Plan9’s Venti service?
• dfdz 5y ago
  > I'm tired creating backups, or loosing data. Let's be honest, sometimes it happens. Does a filesystem that never deletes/overwrites anything exist? I use linux and system link the home folder to a Dropbox folder. There is an extended history option [1], so that you can go back a year for any file. I think it’s a reasonably close solution if you do not mind the cost and syncing to cloud. [1]https://help.dropbox.com/files-folders/restore-delete/extended-version-history https://help.dropbox.com/files-folders/restore-delete/extend...
• wruza 5y ago
  Is Apple Time Machine dead? Many years passed since I used my last one. But anyway, I'm tired creating backups Have you tried to automate them? Thousands of free and paid software to do that. Setup once, forget until next loss.
• mey 5y ago
  On Windows, since 8.1, there is the built in File History. https://support.microsoft.com/en-us/windows/backup-and-restore-in-windows-352091d2-bb9d-3ea3-ed18-52ef2b88cbef https://support.microsoft.com/en-us/windows/backup-and-resto... On macOS, there is Time Machine built in. https://support.apple.com/en-us/HT201250 https://support.apple.com/en-us/HT201250 ZFS does have the concept of Snapshots but I would not consider that user friendly or useful for version tracking. There is plenty of Version Control systems out there, like Git, again not user friendly for what you want. Dropbox/OneDrive/Google Drive provide various levels of version tracking of changes. I personally use Backblaze (30 days, continuous) and Tarsnap (45 days, nightly) for both off-site and temporal recovery when I really need it. Backblaze is used for "desktop" systems, Tarsnap for "server" systems.
  • mikewarot 5y ago
    >On Windows, since 8.1, there is the built in File History. Why am I only hearing about this today? That's an amazing feature to support.
  • hulitu 5y ago
    " Set up a drive for File History Before you start using File History to back up your files, you need to first select where your backups are saved." This is just backup.
• trelliscoded 5y ago
  S3FS with versioning enabled on the S3 bucket would be a pretty low friction way to implement this. You'd also have to enable cross-region replication to deal with your "backups" being in the same place as your working copies. OpenVMS also has a versioning filesystem and a NFS server. The x64 version is available as a private beta, but it's not cheap. Personally, I just keep most stuff in a Google Drive folder and important stuff gets the keep versions forever bit turned on. Everything else goes in a git tree and have some automation to clone it to a reliable utility server when the OS thinks there's plenty of bandwidth available. It's worth noting that the Google Drive solution is the only one that I've been able to get to work reliably across Linux, macOS, Windows, Android, and iOS. I've tried OneDrive, Dropbox, and iCloud Files, but they all have various problems with conflict resolution, annoying upselling, or limited file type support for versioning.
  • alexvoda 5y ago
    S3FS can also be paired with MinIO (1) for a selfhosted solution. MinIO does apear to support versioning. Also it appears VSI will offer free comunity licenses for OpenVMS x64 in the future (3). (1) https://min.io https://min.io (2) https://min.io/product/object-versioning-bucket-versioning https://min.io/product/object-versioning-bucket-versioning (3) https://vmssoftware.com/about/news/2020-07-28-community-license/ https://vmssoftware.com/about/news/2020-07-28-community-lice...
• drpixie 5y ago
  Not exactly what you're asking for, but how about: * use ZFS (or any snapshot capable file system) * trigger snapshots at regular (short) intervals (as chosen by you). ZFS snapshots are very low cost, and you can browse them using your normal file tools. You could reasonably snapshot every (say) 5 minutes, but I wouldn't try every second :)
• potatoalienof13 5y ago
  I think NilFS [0] fits your description, though it is somewhat obscure. It creates a checkpoint with every write, which can mounted after being turned into a snapshot. Although the website seems to have not been updated in a while, it's mailing lists appear to be active. [1] https://nilfs.sourceforge.io/en/ https://nilfs.sourceforge.io/en/ [0] https://marc.info/?l=linux-nilfs https://marc.info/?l=linux-nilfs [1]
• 0des 5y ago
  I think you can do this with fossil and venti
• kadoban 5y ago
  Even if the perfect version of this exists, it will _not_ save you from needing backups. The FS can always screw up or the drive(s) go bad.
• qwerty456127 5y ago
  I'd like the opposite - an FS which really deletes everything you command it to so no recovery tool would find (let alone recover) there was anything.