3 ms·
A thought: How does one apply this to socket-based IO? As far as I can tell, everything involving sockets goes through tons of queues like the ones he describe
by kevingadd 14y ago
A thought: How does one apply this to socket-based IO?
As far as I can tell, everything involving sockets goes through tons of queues like the ones he describes; concurrency, not parallelism.
If I kick off reads on ten different sockets, when the read completes the kernel has to put the thread waiting for the read onto a 'ready to run' queue before any work can actually happen. Scale up to 1000 pending reads and now that queue could get pretty full pretty quick, and lots of data is moving in and out. As I understand things, select and epoll improve on this by making the queue more efficient: now instead of using a thread per socket, and having the OS queue up threads on the ready-to-run queue, the OS helps you manage a queue of 'ready sockets', and you respond to those manually. But there's still a queue!
Are there IO APIs out there in modern OSes that let you reduce queueing in such a way? Like, hypothetically if performing a read on a socket required me to pass in a callback that would process the data from the read, the OS could respond to incoming data on that socket by immediately running the callback, whether it's on an OS-managed user-mode thread pool or via some other mechanism. This would at least reduce the amount of queueing going on - at most you have a queue of tasks for a thread pool that are responding to complete IO (similar to epoll, I guess?) but I bet there are ways to improve on that too.
(If memory serves, IO Completion Ports on NT have functionality that looks like this... but I bet it still involves lots of queueing and scheduling)
- serialx 14y agoIf we just let kernel do the job, it's essentially a queue. If we do the packet processing ourselves, it can be a parallel problem. http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html http://highscalability.com/blog/2013/5/13/the-secret-to-10-m...
- deleted 14y ago[deleted]
- _yosefk 14y agoI think that this angle adds another dimension - dealing with latency and not letting it hurt throughput. The problem could be "embarrassingly parallel" - 10000 processes with 10000 sockets not ever interfering with each other - or it could be concurrent, with a lot of communication between the processes and with event ordering greatly affecting results. Latency would regardless be a problem - the fact that you read from a socket and you get blocked for a while, and can you find something useful to do in the meantime or not, and how easy it will be to get back to what you were doing once the data arrives. In this sense, sockets are not fundamentally different from, say, DRAM latency - it's a much smaller latency, but you still want to do something in the meanwhile.
- sbov 14y agoIt depends upon how big of a picture you want to look at. E.g. if you look down to the hardware level (your single ethernet cable connecting your computer to a router) it is again no longer parallel (from what I recall - it's been a while since I've studied this). In the parallel gifts example, they had to somehow decide who would buy people what, which tends to not be very parallel. If the store has only one entrance then that too turns into a non-parallel part of the problem. If the gifts were mailed there's only 1 mailman who delivers to an area on a given day. And so on. You're probably always going to find resource contention somewhere, it just isn't always something you have to deal with directly when solving your problem.