3 ms·
> WARC is also resistant to hardware errors though, and has this property without requiring constant fsyncing. I just downloaded a sample WARC file to check fo
by kburman 3y ago
> WARC is also resistant to hardware errors though, and has this property without requiring constant fsyncing.
I just downloaded a sample WARC file to check for any checksums to detect bit rot, but I couldn't find one. Can you share any resources I can explore to understand how WARC is more resistant to hardware errors compared to other file formats?
- marginalia_nu 3y agoIt supports hashes of both the record and record+payload, content length for both, and on top of that if a record is bad, you can easily find the next one from the format itself.