4 ms·
Fair. (You can `.getbuffer` but you still have to keep the underlying BytesIO object "open" somehow.) I'm not convinced this is going to bottleneck things, tho
by zahlman 11mo ago
Fair. (You can `.getbuffer` but you still have to keep the underlying BytesIO object "open" somehow.)
I'm not convinced this is going to bottleneck things, though.
(On the flip side, I guess the OS is likely to cache any disk write in memory anyway.)
- carderne 11mo agoI’m just a casual observer of this thread, but I think you’d find it worthwhile to read up a bit on zero-copy stuff. It’s ~impossible in Python (because you don’t control memory) and hard in C/similar (because of use-after-free). Rust’s borrow checker makes it easier, but it’s still tricky (for non-trivial applications). You have to do all your transformations and data movements while only referencing the original data.