3 ms·

> Do not try to snapshot a running database though. Before you execute the snapshot you should make sure everything is flushed to disk. Is a database (or anyth

> Do not try to snapshot a running database though. Before you execute the snapshot you should make sure everything is flushed to disk. Is a database (or anything else) really supposed to behave that way? Shouldn't every state along the way be possible to resume from? In particular, shouldn't database transactions take care of this?

amelius 10y ago
If the snapshot is atomic as it should be, then yes.
shanemhansen 10y ago
A database should handle a snapshot gracefully. I think you should only expect corruption if you are taking a non-atomic backup with something like tar.
majewsky 10y ago
Yes, it is. One of the fundamental principles of practical RDBMS is durability (the D in https://en.wikipedia.org/wiki/ACID https://en.wikipedia.org/wiki/ACID), meaning that when the DB acknowledges a commit to the client, it must be guaranteed that the transaction has been reflected in permanent storage (i.e., on disk). The fine print: Due to the architecture of file system access in contemporary kernels (esp. file system caches), it's next to impossible for a user-space application to actually guarantee that writes are durable (i.e. synced to disk at a defined point in time), but most RDBMS manage well enough in practice.
- cyphar 10y ago
  > The fine print: Due to the architecture of file system access in contemporary kernels (esp. file system caches), it's next to impossible for a user-space application to actually guarantee that writes are durable (i.e. synced to disk at a defined point in time), but most RDBMS manage well enough in practice. While this doesn't relate to a specific file, you can always look at the size of Dirty and Writeback in /proc/meminfo. They correspond to the amount of dirty file page caches that have yet to be synced.
hodgesrm 10y ago
Maybe. RDBMS should be able to recover from any crash consistent copy of storage, where crash consistency means the storage represents a state of the system at run time. FS snapshots provide this property for single volumes. The problem is that production RDBMS tend to span multiple volumes, e.g., for logs vs. data to pick a simple case. FS snapshots in this case may not be consistent which would cause recovery to fail. AFAIK this is why even robust RDBMS like MS SQL server use mechanisms like VSS to ensure data are flushed, which guarantees not just crash consistency but application consistency.

amelius 10y ago

If the snapshot is atomic as it should be, then yes.

shanemhansen 10y ago

A database should handle a snapshot gracefully. I think you should only expect corruption if you are taking a non-atomic backup with something like tar.

majewsky 10y ago

Yes, it is. One of the fundamental principles of practical RDBMS is durability (the D in https://en.wikipedia.org/wiki/ACID https://en.wikipedia.org/wiki/ACID), meaning that when the DB acknowledges a commit to the client, it must be guaranteed that the transaction has been reflected in permanent storage (i.e., on disk). The fine print: Due to the architecture of file system access in contemporary kernels (esp. file system caches), it's next to impossible for a user-space application to actually guarantee that writes are durable (i.e. synced to disk at a defined point in time), but most RDBMS manage well enough in practice.

cyphar 10y ago
> The fine print: Due to the architecture of file system access in contemporary kernels (esp. file system caches), it's next to impossible for a user-space application to actually guarantee that writes are durable (i.e. synced to disk at a defined point in time), but most RDBMS manage well enough in practice. While this doesn't relate to a specific file, you can always look at the size of Dirty and Writeback in /proc/meminfo. They correspond to the amount of dirty file page caches that have yet to be synced.

hodgesrm 10y ago

Maybe. RDBMS should be able to recover from any crash consistent copy of storage, where crash consistency means the storage represents a state of the system at run time. FS snapshots provide this property for single volumes. The problem is that production RDBMS tend to span multiple volumes, e.g., for logs vs. data to pick a simple case. FS snapshots in this case may not be consistent which would cause recovery to fail. AFAIK this is why even robust RDBMS like MS SQL server use mechanisms like VSS to ensure data are flushed, which guarantees not just crash consistency but application consistency.