3 ms·
Yes, you can and should use random file access to load things from a zip file. For example, Go's zip package takes an io.ReaderAt interface, not an io.Reader. U
by skybrian 9y ago
Yes, you can and should use random file access to load things from a zip file. For example, Go's zip package takes an io.ReaderAt interface, not an io.Reader. Using sequential I/O for this would be inefficient.
- jlouis 9y agoThe problem here is that you mostly have a streaming interface In s3.
- skybrian 9y agoI'm not familiar with s3, but it seems like if you want random access for a key-value database, you should store each file under a separate key? Storing an entire file archive as a blob (in any format) and downloading the whole thing just to access one file seems like a weird way to do it. Apparently there is "s3 select" in preview [1], but it only supports gzipped CSV or JSON. [1] https://aws.amazon.com/blogs/aws/s3-glacier-select/ https://aws.amazon.com/blogs/aws/s3-glacier-select/
- etaioinshrdlu 9y agoS3 supports reading ranges of bytes. It might not always be a great idea for performance.