3 ms·
That's pretty interesting. One downside might be losing the ability to do checksums on what's been downloaded, as a safeguard. I wonder if there's a way to pr
by tyingq 5y ago
That's pretty interesting. One downside might be losing the ability to do checksums on what's been downloaded, as a safeguard. I wonder if there's a way to preserve that without ruining the parallel downloads and un-tarring.
- ctecte 5y agoI did look into performing stuff like md5 inline while streaming the download, however it tended to slow down the extraction significantly. Other less rigorous checks like crc32 are still possible inline though with minimal performance reduction. It should also be possible to split off and write the raw download to disk in the background with something like an io.TeeReader object [0]. You could then checksum it after the fact like you normally would. [0]: https://pkg.go.dev/io#TeeReader https://pkg.go.dev/io#TeeReader