4 ms·
Extra copying is the downside if you don't control what you're reading from. Thankfully memcpy is fast but if your input is very big it's still painful. In rust
by pedrocr 6y ago
Extra copying is the downside if you don't control what you're reading from. Thankfully memcpy is fast but if your input is very big it's still painful. In rust I just used an API that allows reading from both an input stream and memory so that if you are reading directly from a file the copy is avoided:
https://github.com/pedrocr/rawloader/blob/c793a132fa03e336f8c5c078f01d3ed1a3ee8c07/src/decoders/mod.rs#L107-L128 https://github.com/pedrocr/rawloader/blob/c793a132fa03e336f8...
- midjji 6y agoIs the extra copying even performed? It either has to be a small thing, or processed in chunks, or oom might be a problem. So its small, in this case, will the compiler really copy, or might it just optimize the operation away?
- pedrocr 6y agoIf you feed it an N byte array it will have to allocate an N+16 array and memcpy. If you're reading from the network/disk/etc then there is no need to copy if you can allocate the larger array directly.