5 ms·

Actually, I have a hard time to understand this. While snapshot reduces file corruptions, it is not a guarantee the best I understand. A corrupted file can man

by liuliu 4y ago

Actually, I have a hard time to understand this. While snapshot reduces file corruptions, it is not a guarantee the best I understand. A corrupted file can manifest itself in many ways. But ultimately, it has to manifest itself as a business logic error, i.e. you increased balance on one entity but didn't on another, causing sum of balances to change (a corruption). Thus, any discussion on file corruptions without a file system that supports transaction, requires every application to use a competent database underneath (SQLite) at least. And even with transactional support in a file system or using a database, you need every application to have the correct business logic that does the transaction correctly as well. All-in-all, correct backup cannot be solved universally without knowing all the applications. The best we can do is to probabilistically avoid obvious issues, i.e. using FS-level snapshot.

cyounkins 4y ago
Yes, you are correct in that it does not guarantee correctness. Point-in-time snapshots are necessary but not sufficient. Without them, the possible corruption scenarios are infinite and cannot be handled or even detected. With them, it is up to the applications to do the right thing in what is equivalent to a power loss event.
rzzzt 4y ago
The Volume Shadow Copy Service mentioned at the end has support for application-level notifications via writers, the most prominent one being SQL Writer: https://docs.microsoft.com/en-us/sql/relational-databases/backup-restore/sql-server-vss-writer-backup-guide?view=sql-server-ver15#sql-writer https://docs.microsoft.com/en-us/sql/relational-databases/ba...
adrianmonk 4y ago
One way of looking at it is that applications already need to be able to survive power loss, a crash of the computer, a crash of the application, or a forcible kill of the application. If they can survive those, they can probably survive a filesystem snapshot being taken at just the wrong moment.
- kazinator 4y ago
  Right! But not necessarily a file by file backup.
kazinator 4y ago
A snapshot captures some state of the filesystem that the filesystem actually had at some point. So when you recover that state, and then run the application, it's similar to the OS having crashed or the power having been lost. Whereas non-snapshot backups don't have that property; the material in the backup is not necessarily identical to any past state of the filesystem that existed. It's something like a past state plus random roll-backs of files, or portions of files, from multiple previous states. Guess which of these situations applications are much more likely to be able to recover from (if any at all)? When people write crash recovery code, they typically assume that the world simply stopped, not that it stopped, and then some data was randomly rewound to unspecified older states. I don't think that you can reasonably defend against data that is restored from a backup, where the oldest part of the backup is an hour older than the newest. A backup is like a raster scan image of a fast moving object. What should be a rectangular train car looks like a parallelogram: it's not a picture of any scene that existed. Imagine that the raster lines are randomly sampled (not a progressive scan, or even interlaced) and now recover a sane image.
- liuliu 4y ago
  I think we need to be a bit more concrete than analogies for my brain to process. So here we go: If your application relies on multi-file collaborated persisted states (for example, an append-only log and a database snapshot), application can make reasonable assumptions on when a file state is committed, such as a database snapshot is `fsync`ed before the append-only log started. Ideally, even on a filesystem without transaction support, this order is preserved in time. However, it may not be preserved for a "file-to-file" backup system because it can loop over the append-only log file first before the database snapshot, causing ordering issues. That could result a "ABA" problem, where the append-only log is corresponding to an older database snapshot, and potentially causes issues. That has been said, is this a common scenario (multi-file persisted state) for applications? (I believe SQLite handles this particular ordering issue fine). I am not sure, and just want to call out filesystem snapshot solves a very particular problem. If your application relies only one file for the state, or these files are orthogonal to each other (.xlsx or .docx or any .markdown, .mp4, .jpg files), it is a non-issue. And if you need to backup a database, you better do that with the database provided tools.
a1369209993 4y ago
> While snapshot reduces file corruptions, it is not a guarantee the best I understand. The idea is that you'll only encounter corruptions that the application could already hit due to crashes or power outages (and hence hopefully supports recovering from). For example, with naive reads, you might: read the first half of the file context switch to the application application writes to the first half of the file application fsyncs previous writes application writes to the second half of the file context switch back to you read the second half of the file and end up with data in the second half of the file that the application normally only writes after it's sure that corresponding data has been written to the first half.

by liuliu 4y ago

cyounkins 4y ago

Yes, you are correct in that it does not guarantee correctness. Point-in-time snapshots are necessary but not sufficient. Without them, the possible corruption scenarios are infinite and cannot be handled or even detected. With them, it is up to the applications to do the right thing in what is equivalent to a power loss event.

rzzzt 4y ago

The Volume Shadow Copy Service mentioned at the end has support for application-level notifications via writers, the most prominent one being SQL Writer: https://docs.microsoft.com/en-us/sql/relational-databases/backup-restore/sql-server-vss-writer-backup-guide?view=sql-server-ver15#sql-writer https://docs.microsoft.com/en-us/sql/relational-databases/ba...

adrianmonk 4y ago

One way of looking at it is that applications already need to be able to survive power loss, a crash of the computer, a crash of the application, or a forcible kill of the application. If they can survive those, they can probably survive a filesystem snapshot being taken at just the wrong moment.

kazinator 4y ago
Right! But not necessarily a file by file backup.

kazinator 4y ago

A snapshot captures some state of the filesystem that the filesystem actually had at some point. So when you recover that state, and then run the application, it's similar to the OS having crashed or the power having been lost. Whereas non-snapshot backups don't have that property; the material in the backup is not necessarily identical to any past state of the filesystem that existed. It's something like a past state plus random roll-backs of files, or portions of files, from multiple previous states. Guess which of these situations applications are much more likely to be able to recover from (if any at all)? When people write crash recovery code, they typically assume that the world simply stopped, not that it stopped, and then some data was randomly rewound to unspecified older states. I don't think that you can reasonably defend against data that is restored from a backup, where the oldest part of the backup is an hour older than the newest. A backup is like a raster scan image of a fast moving object. What should be a rectangular train car looks like a parallelogram: it's not a picture of any scene that existed. Imagine that the raster lines are randomly sampled (not a progressive scan, or even interlaced) and now recover a sane image.

liuliu 4y ago
I think we need to be a bit more concrete than analogies for my brain to process. So here we go: If your application relies on multi-file collaborated persisted states (for example, an append-only log and a database snapshot), application can make reasonable assumptions on when a file state is committed, such as a database snapshot is `fsync`ed before the append-only log started. Ideally, even on a filesystem without transaction support, this order is preserved in time. However, it may not be preserved for a "file-to-file" backup system because it can loop over the append-only log file first before the database snapshot, causing ordering issues. That could result a "ABA" problem, where the append-only log is corresponding to an older database snapshot, and potentially causes issues. That has been said, is this a common scenario (multi-file persisted state) for applications? (I believe SQLite handles this particular ordering issue fine). I am not sure, and just want to call out filesystem snapshot solves a very particular problem. If your application relies only one file for the state, or these files are orthogonal to each other (.xlsx or .docx or any .markdown, .mp4, .jpg files), it is a non-issue. And if you need to backup a database, you better do that with the database provided tools.

a1369209993 4y ago

> While snapshot reduces file corruptions, it is not a guarantee the best I understand. The idea is that you'll only encounter corruptions that the application could already hit due to crashes or power outages (and hence hopefully supports recovering from). For example, with naive reads, you might: read the first half of the file context switch to the application application writes to the first half of the file application fsyncs previous writes application writes to the second half of the file context switch back to you read the second half of the file and end up with data in the second half of the file that the application normally only writes after it's sure that corresponding data has been written to the first half.