4 ms·
My own experience is that nameless temporary files (eg tempfile.TemporaryFile()) are _considerably_ faster than (c)StringIO and worth using whenever the "file"
by calpaterson 5y ago
My own experience is that nameless temporary files (eg tempfile.TemporaryFile()) are _considerably_ faster than (c)StringIO and worth using whenever the "file" is bigger than a few bytes.
- klmr 5y agoOdd, why would this be? Read and write operations on temporary files need to go through the kernel and thus lead to frequent context switches. The same shouldn’t be true for a string buffer except on reallocation.
- WGH_ 5y agoMaybe page cache (the thing tmpfs relies on) doesn't do full copy when reallocation, but rather, roughly speaking, appends extra pages to some sort of linked list? It would explain why it's faster.
- mixmastamyk 5y agoIf that’s the case, there’s a bug somewhere.
- calpaterson 5y agoI'm sure I must have made a mistake as I wasn't able to replicate this with some quick local benchmarking.