4 ms·
The benefit is that you're left with a database state that actually existed; there's no guarantee from the database's perspective that dropping some committed t
by cwillu 1y ago
The benefit is that you're left with a database state that actually existed; there's no guarantee from the database's perspective that dropping some committed transactions and not others that came after will result in a valid state.
- HelloNurse 1y agoThis is the main point that the OP misses: even if the newer portion of the WAL file isn't corrupted, its content cannot be used in any way because doing so would require the lost transactions from the corrupted block. The chained checksums are a feature, not gratuitous fragility.
- AlotOfReading 1y agoSqlite could attempt to recover the detected errors though and not lose the transactions.
- hobs 1y agoAnd not get in an infinite loop, and not harm the startup time of the process inordinately, and... This is just basically how a WAL works, if you have an inconsistent state the transaction is rolled back - at that point you need to redo your work.
- HelloNurse 1y agoThe WAL can be replayed up to the first corrupted block, period. It's like a rockslide destroying a road: you can progress up to the gap or rubble heap, but you cannot expect to use the unharmed road past the insurmountable obstacle.
- daneel_w 1y agoThe WAL was corrupted, the actual data is lost. There's no parity. You're suggesting that sqlite should somehow recreate the data from nothing.
- avinassh 1y ago> The WAL was corrupted, the actual data is lost. There's no parity. You're suggesting that sqlite should somehow recreate the data from nothing. Not all frames in the WAL are important. Sure, recovery may be impossible in some cases, but not all checksum failures are impossible to recover from.
- pests 1y ago> but not all checksum failures are impossible to recover from Which failures are possible to recover from?
- dec0dedab0de 1y agowhat if the corruption only affected the stored checksum, but not the data itself?
- jandrewrogers 1y agoThere are two examples I know of that require no additional data: First, force a re-read of the corrupted page from disk. A significant fraction of data corruption occurs while it is being moved between storage and memory due to weak error detection in that part of the system. A clean read the second time would indicate this is what happened. Second, do a brute-force search for single or double bit flips. This involves systematically flipping every bit in the corrupted page, recomputing the checksum, and seeing if corruption is detected.
- teraflop 1y agoYes, and it's not just about application-level integrity. The WAL operates at a page level, so dropping one WAL entry and then applying later ones would be likely to cause corruption at the B-tree level. For instance, say you have a node A which has a child B: * Transaction 1 wants to add a value to B, but it's already full, so B is split into new nodes C and D. Correspondingly, the pointer in A that points to B is removed, and replaced with pointers to C and D. * Transaction 2 makes an unrelated change to A. If you skip the updates from transaction 1, and apply the updates from transaction 2, then suddenly A's data is overwritten with a new version that points to nodes C and D, but those nodes haven't been written. The pointers just point to uninitialized garbage.