4 ms·
I really don't like checksums. The reason the author gives is twofold: detect things like CRLF conversions and bit rot. For the former, a static code that incl
by lucb1e 7y ago
I really don't like checksums. The reason the author gives is twofold: detect things like CRLF conversions and bit rot.
For the former, a static code that includes a CRLF does just fine and allows modifying the file without having to write code to recompute your annoying checksum.
For the latter, I find it questionable whether it's the file format's responsibility to prevent bit rot. What's it going to do anyway? Reject the file? Knowing whether the file is undamaged is good, but most of the time people will want it to do a best effort attempt at reading it anyway, and knowing whether integrity has been maintained is actually not that important for most files (if it is, it's usually a cryptographic feature and you'll not want to use CRC32 or MD5 as the author suggests). To recover from bit rot, you'll need a backup (or an error correction scheme but that seems out of scope of this article). If every file format has its own checksum, your backup solution can't check the files for integrity anyway, so it seems to me like it makes more sense to have checksumming be a filesystem feature.
I'd like it if checksums could just not be a thing in file headers anymore. The number of times it helped me is exactly zero, but I did run into issues when wanting to open an old file or modify a file. I would argue checksums prevent hacking in the HN sense of the word.
- joosters 7y agoChecksums are also a pain for larger files. If a 4GB movie has a checksum, is the app supposed to read all the file in to calculate and validate the checksum prior to playing the movie? That’s inefficient. It also doesn’t help programs dealing with bad data. Apps still must handle invalid files gracefully, even if the checksum is good. Otherwise, hackers can create malicious invalid files with correct checksums. Checksums also make small edits to a file painful, since every program must now know how to update the checksums. Imagine if text files had checksums - how could you possibly do simple regex replacements, e.g. using ‘sed’?
- pharrington 7y agoRegarding your second thought—one obvious way a checksum helps in gracefully handling invalid files is that a checksum pass/fail decisively rules out with a problem occurred during the data transfer. For example, a checksum always you to know when it is/isn't appropriate to automatically attempt a redownload or tell the user to try to redownload a file.
- deleted 7y ago[deleted]
- colejohnson66 7y ago> Checksums are also a pain for larger files. If a 4GB movie has a checksum, is the app supposed to read all the file in to calculate and validate the checksum prior to playing the movie? That’s inefficient. The page did mention doing it for the header, not the whole file