6 ms·
> Now the question is why to bother with multiprocess socket servers at all - aren't threads and events better? There's at least one good niche for them - dynam
by rdtsc 13y ago
> Now the question is why to bother with multiprocess socket servers at all - aren't threads and events better? There's at least one good niche for them - dynamic languages like Python or Ruby, which need multiple OS processes to achieve real concurrency [my emphasis]
That is not true. It is an often repeated misconception. It makes it sound like Python creators were just incompetent and just stuck threads in there even though they are completely useless. In fact Python's threads work well for IO concurrency. I used them and saw great speedup when accepting and handling simultaneous socket connections. Yes you won't get CPU concurrency, but if your server is not CPU bound you might not notice much of a difference.
IO concurrency is real concurrency. In 8 years using Python for fun and professionally I probably wrote more IO concurrent code than CPU concurrent code. Even then for CPU concurrent code I would have had to drop into C using an extension (and there you can release the GIL anyway).
Now, the obvious follow up is that in case of IO concurrency you are often better of using gevent or eventlet. You get lighter weight threads (memory wise) and less chances of synchronizations bugs (since greenlet based green threads will switch only on IO concurrency points, socket reads, sleep and explicit waits on green semaphores and locks).
- sluukkonen 13y agoWhat he means is true parallelism (although alternate ruby and python implementations have it).
- rdtsc 13y agoDownloading 2 web pages at the same time without one blocking another from completing is true parallelism. The request is sent for one, while it is in progress (maybe server is slow), another one can go out and come back with data. This can happen for hundreds or thousands of them. These are executed in parallel. So we got concurrent units of work executed at the same time, I fail to see how that is not parallel. Now this is IO concurrency but it is real concurrency. Adding CPU concurrency would be very nice. It might speed things up a bit, or it might not. It really depends. As an example consider haproxy. The little proxy that could. It handles large amounts of concurrent connection in parallel and it is single threaded in its default configuration. I've heard of 100k connections. It deals with IO concurrency. Chances are, making it multi-threaded might not dramatically improve its performance (it might even slow it down).
- meowface 13y agoConcurrent generally means "occurring or able to occurr at the exact same time." So you do have parallel and asynchronous IO, but not concurrent. There's lots of debate over the terms, but I think that's the agreed "exact definition." Of course, does this really make a difference for network IO? Almost always the answer is no. The difference will be on the order of microseconds, maybe milliseconds.
- rdtsc 13y agoMy definition is this: * Concurrency is a property of relationships between tasks in the problem (or the algorithm). Is fetching one page for example independent of another one. If could be so it is concurrent, but it also might not be true, if it is a child page. You have to fetch one page, look at links and then fetch those pages. To the tasks have hard coded sequence so those are not concurrent. * Parallelism is how that algorithm or problem is solved or executed. It could be that you can execute all concurrent units at the same time so you achieve parallelism, which is great. Or it could be that due to a particular architecture or other reasons you execute it serially. Maybe you just have a while loop and fetch one page, wait fetch another one. The problem is concurrent but it is not run in parallel. Notice my definition doesn't include CPU or IO in there. In real world there is both. CPU concurrency interleaved with IO concurrency. That you can then end up running none, one or both in parallel when you execute.
- skyraider 13y agoTo simplify: Concurrent tasks complete in the same, overlapping time period. Parallel tasks run at literally the same time.
- scott_s 13y agoYes. And this implies that concurrent is not necessarily parallel, but parallel is always concurrent. A way to keep the notions straight is that if you're implementing a kernel that will only ever run on one core, you still have to worry about concurrency: all processes, including the kernel, are time-sharing the single core.
- anaphor 13y agoThanks, people need to learn the difference between concurrency and parallelism ffs, I'm tired of explaining the difference all the time.
- RivieraKid 13y agoIt doesn't really matter how you call it. There are more important things people need learn about.
- anaphor 13y agoActually it is important to learn about, because it's possible to do parallelism without bringing in the issues of concurrency (non-determinism, synchronization) and thus write code with less potential for serious bugs.
- RivieraKid 13y agoLearning the difference between c. and p. won't help me write less buggy software. And I think there's no clear consensus what exactly is the difference.
- Peaker 13y agoIt is true, however, that knowing about parallelism without (visible) concurrency is useful for writing less buggy software.
- smegel 13y ago> In fact Python's threads work well for IO concurrency. So long as you don't have any CPU bound threads competing for the GIL ;)
- rbanffy 13y agoThat's why profiling your code is so important. If you have a CPU bound thread, it may be worth to pay the performance penalty of separating some of the program flow in different processes.
- RivieraKid 13y agoBy real concurrency he clearly meant CPU concurrency.
- rdtsc 13y agoI got that, but pretended to not understand in order to illustrate a point. That is a common misconception. And it seems to me nowadays most concurrency people deal with (at least when it comes to server and web back-end world) is heavier IO bound. Yet everyone automatically default to their CS 102 -- algorithms class when they think about solving graph problems in parallel or multiplying matrices. So concurrency automatically is implied to be CPU concurrency.
- nshepperd 13y agoI came in here expecting to see a pedantic debate about the use of the word "concurrency", and I was not disappointed. Though of course you have a point.
- deleted 13y ago[deleted]
- bad_user 13y ago> you won't get CPU concurrency, but if your server is not CPU bound you might not notice much of a difference That's not true. Most applications, with the possible exceptions of proxies, are also CPU bound. Take for example a web service that receives JSON documents. The act of parsing JSON documents is CPU bound. The act of creating a response is CPU bound. In between you can also have IO bound operations, like fetching data from a MySQL database or a Memcached instance, however in the process of creating the final response you also need to transform the data received and that's also CPU bound. As a real world example, I worked on a web-service written in Scala and running on the JVM. Initially it was running on only 8 Heroku dynos and these instances were receiving over 30,000 requests per second of real traffic. These Heroku instances are of course under-powered, because on my modest laptop the same web server is able to handle more than 10,000 requests per second. And yes, asynchronous I/O lets you easily have 100,000 connections per server. But if you need throughput, then the CPU starts being a bottleneck. Of course, my problem with Python and why I migrated away from it is that in truth Python sucks for asynchronous I/O too. But that's another story.
- illumen 13y agoWrong. Those examples are IO bound - memory. You'll find the CPU waiting on memory in these cases. JSON is parsed in C with CPython, or in assembly with PyPy. As a real world example, I've done realtime image processing of a gigabyte per second worth of data on a single machine with asynchronous python. It was IO bound, we had more CPU to spare. Hell, we even had some GPUs sitting there not doing anything because they weren't needed. If you're doing real performance computing, then taking advantage of GPUs/DSP or other hardware is where it is at anyway. Python is quite good at a glue language for interfacing to these things.
- deleted 13y ago[deleted]
- ithkuil 13y agoUnfortunately if the CPU is waiting for memory, then thread is stuck, even if the hardware CPU had resources to execute some other instructions it cannot execute them from the same stream of instructions. Instead it could execute another stream of instructions, i.e. another thread. This is how hyperthreading works and still requires you to write a multithreaded application with the same problems and requirements.