3 ms·
How do they checksum 9GB of data and still boot promptly?
by jeffbee 3y ago
How do they checksum 9GB of data and still boot promptly?
- deleted 3y ago[deleted]
- sillywalk 3y agoFTA: "This is the ingenious part: verification carries little overhead, as it runs as a rolling process once the top-level ‘seal’ has been verified. So verification is a continuous process as the SSV is mounted and accessed."
- vlovich123 3y agoWhich presumably means Merkel Trees.
- droffel 3y agoMerkle trees! You (simplified) hash the data at the leaves of the tree, merge adjacent leaves (concatenation for example), and hash the new leaves, until you're left with just the Merkle Root, a single hash representing the entirety of your data. Verifying the root is easier than loading the data itself into memory and hashing it there since you can verify it piecemeal and without loading it all into memory at once.
- krackers 3y agoBut regardless of the ordering which you read, don't you ultimately still need to read every bit once?
- CJefferson 3y agoEach level of the tree contains a hash of the hashes from the level below, so to verify the top hash you need only hash all its children (which are hashes themselves). Then you can explore a child, continuing recursively. At the bottom you find a hash of an actual file (or part of a file), then hash the file to check it’s validity. This allows you to only hash those parts of the tree you actually want to read.
- viraptor 3y ago> Big Sur, Monterey or Ventura in default Full Security mode has verified every last bit of their 9 GB SSV. In this case they're not selective, so the tree approach doesn't save the reads. (Just makes it easier to identify which part failed)
- vlovich123 3y agoIf you want to verify the entire image, I don’t think you can get around reading the entire image. Because any part you didn’t read to verify the hash is a part that could contain corruption.
- nomel 3y agoThat's around 3s, for their slowest offering.
- w-m 3y agoDon’t these machines have read speeds of like 5000 MB/s? So just read the 9 GB blockwise and apply a super simple hash. Should be done in two seconds.
- viraptor 3y agoIf I include the fs overhead, I can shasum a single 23GB file (likely not fragmented) in 46s. That's uncached, so it's not that quick at boot. 9GB would take ~19s then. Block would go faster, but not an order of magnitude faster.