3 ms·
The better way of ‘overwriting’ is to write the new file under a new name, and then atomically move it to where the old file was. If you have to overwrite (e.g.
by Someone 7y ago
The better way of ‘overwriting’ is to write the new file under a new name, and then atomically move it to where the old file was. If you have to overwrite (e.g. because of lack of disk space), the program that rewrites that file should open it in exclusive mode.
A back-up tool should open files to backup in a mode that disallows others from writing to it, and retry that a few times, each time after waiting a few seconds.
If that keeps failing, there’s a problem. Do you skip backing up the file or retry With a “it’s fine if others are reading” mode, with the risk of getting a hybrid of multiple versions? There is no fully correct answer there.
And it gets worse. Not all programs write files in exclusive mode. Some might even open a file multiple times while rewriting it. Also, consistency may work cross-files, for example with Mac OS packages.
And then, of course, there’s anti-virus tools. They often work below the file system, reading and writing files without the OS being aware of it.
The reason that backup tools work relatively reliable and speedy at all mostly is that most programs that write to files will either regularly close them, or just append to them.
Databases are the exception. They tend to keep files permanently open and regularly write to them. If you use SQLite in a ’normal’ application for user data, you should flush data after (about) every write (if you use it for caching, you can be a bit more loose, as its relatively fine to discard the database when it gets corrupted.
- justinclift 7y ago> A back-up tool should open files to backup in a mode that disallows others from writing to it, and retry that a few times, each time after waiting a few seconds. Generally the preferred approach for backup programs is to trigger a file system snapshot - after quiescing the disks - and back that up. Sometimes (eg databases), the backup program is able to run application specific logic that works in with this (eg flush all database buffers before the snapshot). For the filesystems that don't offer snapshots, some still have the ability to freeze the file system to allow a consistent backup.
- GlitchMr 7y agoAs I said in a comment you responded to. > Programs can deal with the issue by performing atomic full writes by performing renames, but most programs don't bother to. SQLite cannot do that, because it cannot afford doing full writes as databases tend to store a lot of data. I'm aware you can do that, but most programs aren't. If you pick any random program, chances are it won't do that.