3 ms·
If you are writing sequentially to a local disk, perhaps it would make sense to create a tiny shim process that listens on a socket and writes to a file. You co
by twic 9y ago
If you are writing sequentially to a local disk, perhaps it would make sense to create a tiny shim process that listens on a socket and writes to a file. You could do that in C, or perhaps even as a socat invocation.
- atemerev 9y agoInteresting idea! Thank you for the suggestion.
- derefr 9y agoDo note that this should have about equivalent performance to Erlang's disk IO BIFs, as this is essentially what they're already doing—the async IO threads are the "tiny shim process." (The difference mostly comes down to the VM having an efficient internal IPC protocol to them, and the runtime being able to control their CPU core affinity.) Note also that if you turn ERTS's async IO threads off (set the number of them to 0), you should get improved throughput on disk IO tasks, as you're then forcing the scheduler threads to do the disk IO themselves. Of course, this trades off against latency, because the fallback here is blocking calls. Sadly, the async-thread-pool architecture has meant Erlang has had no reason to implement anything like Linux's kernel AIO support. (I wonder if they'd accept a patch that specialized the efile driver for Linux, the way it uses overlapped IO on win32...)