6 ms·

What he means is true parallelism (although alternate ruby and python implementations have it).

rdtsc 13y ago
Downloading 2 web pages at the same time without one blocking another from completing is true parallelism. The request is sent for one, while it is in progress (maybe server is slow), another one can go out and come back with data. This can happen for hundreds or thousands of them. These are executed in parallel. So we got concurrent units of work executed at the same time, I fail to see how that is not parallel. Now this is IO concurrency but it is real concurrency. Adding CPU concurrency would be very nice. It might speed things up a bit, or it might not. It really depends. As an example consider haproxy. The little proxy that could. It handles large amounts of concurrent connection in parallel and it is single threaded in its default configuration. I've heard of 100k connections. It deals with IO concurrency. Chances are, making it multi-threaded might not dramatically improve its performance (it might even slow it down).
- meowface 13y ago
  Concurrent generally means "occurring or able to occurr at the exact same time." So you do have parallel and asynchronous IO, but not concurrent. There's lots of debate over the terms, but I think that's the agreed "exact definition." Of course, does this really make a difference for network IO? Almost always the answer is no. The difference will be on the order of microseconds, maybe milliseconds.
  - rdtsc 13y ago
    My definition is this: * Concurrency is a property of relationships between tasks in the problem (or the algorithm). Is fetching one page for example independent of another one. If could be so it is concurrent, but it also might not be true, if it is a child page. You have to fetch one page, look at links and then fetch those pages. To the tasks have hard coded sequence so those are not concurrent. * Parallelism is how that algorithm or problem is solved or executed. It could be that you can execute all concurrent units at the same time so you achieve parallelism, which is great. Or it could be that due to a particular architecture or other reasons you execute it serially. Maybe you just have a while loop and fetch one page, wait fetch another one. The problem is concurrent but it is not run in parallel. Notice my definition doesn't include CPU or IO in there. In real world there is both. CPU concurrency interleaved with IO concurrency. That you can then end up running none, one or both in parallel when you execute.
    - skyraider 13y ago
      To simplify: Concurrent tasks complete in the same, overlapping time period. Parallel tasks run at literally the same time.
      - scott_s 13y ago
        Yes. And this implies that concurrent is not necessarily parallel, but parallel is always concurrent. A way to keep the notions straight is that if you're implementing a kernel that will only ever run on one core, you still have to worry about concurrency: all processes, including the kernel, are time-sharing the single core.
        barrkel 13y ago
        parallel is always concurrent I don't really agree. The heart of the problem of concurrency is non-determinism, but it's perfectly possible to have deterministic parallel algorithms. Normally the key is not letting the parallel operations interact with one another. So to me, a useful (for discussion) definition of concurrency involves multiple logical tasks, overlapping in time, and interacting with one another, in a non-deterministic way. Whereas parallelism is concerned with taking advantage of physical hardware that can do more than one thing simultaneously. And concurrency does not imply parallelism, nor does parallelism imply concurrency, under my understanding. In particular, data parallelism like SIMD or CUDA is not concurrent.
        rdtsc 13y ago
        > So to me, a useful (for discussion) definition of concurrency involves multiple logical tasks, overlapping in time, and interacting with one another, in a non-deterministic way. Hmm, I would think it would be the opposite, they're concurrent precisely because they don't have to interact. They can run independently. 2 requests from a server are concurrent because they don't have to know about each other and don't have to interact with each. This is a property of the problem domain (idealized web requests) this doesn't tell us anything about how they'll run (in parallel or not). > Whereas parallelism is concerned with taking advantage of physical hardware that can do more than one thing simultaneously. I agree with that. > And concurrency does not imply parallelism, nor does parallelism imply concurrency, under my understanding. In particular, data parallelism like SIMD or CUDA is not concurrent. Don't quite agree with that and don't see why SIMD algorithms have to be a special case. Maybe you compute a dot product between 2 vectors. If you write the algorithm down you have a bunch of multiplications and a sum. You notice that it has a lot of concurrency (the algorithm). If you don't have SIMD you could spawn a thread to multiply out each pair and then to sum. That would be silly. But you'd run in parallel. You could just do it sequentially with a for loop. But if you have SIMD, it know how to run those concurrent algorithmic steps in parallel.
  - plainOldText 13y ago
    I think a lot of people confuse parallelism with concurrency. The easiest analogy I can think of is this: 1. Concurrent means having two cups of water, one in each hand, and drinking(think CPU computation) a little bit from one, then switch to the other. While you drink from a cup someone is filling up the other (think socket IO) 2. Parallel means having two cups, one in each hand,and lifting them up and drinking from them at the same exact time.
    - nknighthb 13y ago
      http://www.merriam-webster.com/dictionary/concurrent http://www.merriam-webster.com/dictionary/concurrent http://en.wiktionary.org/wiki/concurrent http://en.wiktionary.org/wiki/concurrent http://en.wikipedia.org/wiki/Concurrency_(computer_science) http://en.wikipedia.org/wiki/Concurrency_(computer_science) To state the obvious, you're attempting to make distinctions that either don't exist, or do not have a consensus. You need to find new words. Your definition of concurrent is just... wrong.
      - anaphor 13y ago
        You're the one using the wrong words: actual experts on concurrency and parallelism disagree with you: http://existentialtype.wordpress.com/2011/03/17/parallelism-is-not-concurrency/ http://existentialtype.wordpress.com/2011/03/17/parallelism-... http://blog.golang.org/concurrency-is-not-parallelism http://blog.golang.org/concurrency-is-not-parallelism http://ghcmutterings.wordpress.com/2009/10/06/parallelism-concurrency/ http://ghcmutterings.wordpress.com/2009/10/06/parallelism-co...
        nknighthb 13y ago
        Oh? Then why don't they say anything in their posts that disagrees with my view on the subject? (Hint: You're replying to the first comment I made in this thread, so the idea that I used the words "concurrent" or "parallel" in any particular way, much less wrongly, is objectively incorrect.)
      - plainOldText 13y ago
  - vidarh 13y ago
    It is very common to describe pre-emptive multitasking on a single core as concurrent processing. Your definition is a general dictionary definition - it does not necessarily fit well with usage in technology.
    - meowface 13y ago
      You're right, both "parallel" and "concurrent" are ambiguous if using the dictionary definitions. Computer science has added somewhat new definitions to both of those terms but they're not really universally known or understood. It'd be nice if new terms were used entirely, really.
  - revelation 13y ago
    That definition makes very little sense for computing. The basis of multithreading is an illusion of concurrency.
    - deleted 13y ago
      [deleted]
    - eropple 13y ago
      Unless you have multiple cores.
    - barrkel 13y ago
      No, multithreading is an OS abstraction providing the illusion of parallelism - which might actually be parallel, if more than one hardware executor is available - and is an implementation technique for concurrency, but not the only one.
- iooi 13y ago
  > So we got concurrent units of work executed at the same time, I fail to see how that is not parallel. I thought that by definition this is impossible under the GIL. Not completely sure, but would love to know. I have written thousands of lines using gevent and eventlet but have only achieved peaks of 10 Mb/s (on servers that have at least 100), and I'm sure that truly concurrent languages could fully take advantage of that throughput -- currently in the process of migrating from Python.
  - rdtsc 13y ago
    It is hard to tell but if you mostly fetch data without processing it, copying it, compute it, then gevent can certainly handle concurrent socket operations. Python's GIL won't let you execute Python code in parallel like say you start multiplying numbers in one thread and another. You won't multiply twice as many numbers because of the GIL. But for IO concurrency you should achieve parallelism (unless you have a string CPU consuming part in there as well).
  - ddorian43 13y ago
    what language are you migrating to ?
  - meowface 13y ago
    If you have anything that's even ever so briefly blocking gevent, then you will indeed be bottlenecked in that way. I'd be curious to see if your migration does allow better bandwidth peaks, though.

rdtsc 13y ago

Downloading 2 web pages at the same time without one blocking another from completing is true parallelism. The request is sent for one, while it is in progress (maybe server is slow), another one can go out and come back with data. This can happen for hundreds or thousands of them. These are executed in parallel. So we got concurrent units of work executed at the same time, I fail to see how that is not parallel. Now this is IO concurrency but it is real concurrency. Adding CPU concurrency would be very nice. It might speed things up a bit, or it might not. It really depends. As an example consider haproxy. The little proxy that could. It handles large amounts of concurrent connection in parallel and it is single threaded in its default configuration. I've heard of 100k connections. It deals with IO concurrency. Chances are, making it multi-threaded might not dramatically improve its performance (it might even slow it down).

meowface 13y ago
Concurrent generally means "occurring or able to occurr at the exact same time." So you do have parallel and asynchronous IO, but not concurrent. There's lots of debate over the terms, but I think that's the agreed "exact definition." Of course, does this really make a difference for network IO? Almost always the answer is no. The difference will be on the order of microseconds, maybe milliseconds.
- rdtsc 13y ago
  My definition is this: * Concurrency is a property of relationships between tasks in the problem (or the algorithm). Is fetching one page for example independent of another one. If could be so it is concurrent, but it also might not be true, if it is a child page. You have to fetch one page, look at links and then fetch those pages. To the tasks have hard coded sequence so those are not concurrent. * Parallelism is how that algorithm or problem is solved or executed. It could be that you can execute all concurrent units at the same time so you achieve parallelism, which is great. Or it could be that due to a particular architecture or other reasons you execute it serially. Maybe you just have a while loop and fetch one page, wait fetch another one. The problem is concurrent but it is not run in parallel. Notice my definition doesn't include CPU or IO in there. In real world there is both. CPU concurrency interleaved with IO concurrency. That you can then end up running none, one or both in parallel when you execute.
  - skyraider 13y ago
    To simplify: Concurrent tasks complete in the same, overlapping time period. Parallel tasks run at literally the same time.
    - scott_s 13y ago
      Yes. And this implies that concurrent is not necessarily parallel, but parallel is always concurrent. A way to keep the notions straight is that if you're implementing a kernel that will only ever run on one core, you still have to worry about concurrency: all processes, including the kernel, are time-sharing the single core.
      - barrkel 13y ago
        parallel is always concurrent I don't really agree. The heart of the problem of concurrency is non-determinism, but it's perfectly possible to have deterministic parallel algorithms. Normally the key is not letting the parallel operations interact with one another. So to me, a useful (for discussion) definition of concurrency involves multiple logical tasks, overlapping in time, and interacting with one another, in a non-deterministic way. Whereas parallelism is concerned with taking advantage of physical hardware that can do more than one thing simultaneously. And concurrency does not imply parallelism, nor does parallelism imply concurrency, under my understanding. In particular, data parallelism like SIMD or CUDA is not concurrent.
        rdtsc 13y ago
        > So to me, a useful (for discussion) definition of concurrency involves multiple logical tasks, overlapping in time, and interacting with one another, in a non-deterministic way. Hmm, I would think it would be the opposite, they're concurrent precisely because they don't have to interact. They can run independently. 2 requests from a server are concurrent because they don't have to know about each other and don't have to interact with each. This is a property of the problem domain (idealized web requests) this doesn't tell us anything about how they'll run (in parallel or not). > Whereas parallelism is concerned with taking advantage of physical hardware that can do more than one thing simultaneously. I agree with that. > And concurrency does not imply parallelism, nor does parallelism imply concurrency, under my understanding. In particular, data parallelism like SIMD or CUDA is not concurrent. Don't quite agree with that and don't see why SIMD algorithms have to be a special case. Maybe you compute a dot product between 2 vectors. If you write the algorithm down you have a bunch of multiplications and a sum. You notice that it has a lot of concurrency (the algorithm). If you don't have SIMD you could spawn a thread to multiply out each pair and then to sum. That would be silly. But you'd run in parallel. You could just do it sequentially with a for loop. But if you have SIMD, it know how to run those concurrent algorithmic steps in parallel.
- plainOldText 13y ago
  I think a lot of people confuse parallelism with concurrency. The easiest analogy I can think of is this: 1. Concurrent means having two cups of water, one in each hand, and drinking(think CPU computation) a little bit from one, then switch to the other. While you drink from a cup someone is filling up the other (think socket IO) 2. Parallel means having two cups, one in each hand,and lifting them up and drinking from them at the same exact time.
  - nknighthb 13y ago
    http://www.merriam-webster.com/dictionary/concurrent http://www.merriam-webster.com/dictionary/concurrent http://en.wiktionary.org/wiki/concurrent http://en.wiktionary.org/wiki/concurrent http://en.wikipedia.org/wiki/Concurrency_(computer_science) http://en.wikipedia.org/wiki/Concurrency_(computer_science) To state the obvious, you're attempting to make distinctions that either don't exist, or do not have a consensus. You need to find new words. Your definition of concurrent is just... wrong.
    - anaphor 13y ago
      You're the one using the wrong words: actual experts on concurrency and parallelism disagree with you: http://existentialtype.wordpress.com/2011/03/17/parallelism-is-not-concurrency/ http://existentialtype.wordpress.com/2011/03/17/parallelism-... http://blog.golang.org/concurrency-is-not-parallelism http://blog.golang.org/concurrency-is-not-parallelism http://ghcmutterings.wordpress.com/2009/10/06/parallelism-concurrency/ http://ghcmutterings.wordpress.com/2009/10/06/parallelism-co...
      - nknighthb 13y ago
        Oh? Then why don't they say anything in their posts that disagrees with my view on the subject? (Hint: You're replying to the first comment I made in this thread, so the idea that I used the words "concurrent" or "parallel" in any particular way, much less wrongly, is objectively incorrect.)
    - plainOldText 13y ago
- vidarh 13y ago
  It is very common to describe pre-emptive multitasking on a single core as concurrent processing. Your definition is a general dictionary definition - it does not necessarily fit well with usage in technology.
  - meowface 13y ago
    You're right, both "parallel" and "concurrent" are ambiguous if using the dictionary definitions. Computer science has added somewhat new definitions to both of those terms but they're not really universally known or understood. It'd be nice if new terms were used entirely, really.
- revelation 13y ago
  That definition makes very little sense for computing. The basis of multithreading is an illusion of concurrency.
  - deleted 13y ago
    [deleted]
  - eropple 13y ago
    Unless you have multiple cores.
  - barrkel 13y ago
    No, multithreading is an OS abstraction providing the illusion of parallelism - which might actually be parallel, if more than one hardware executor is available - and is an implementation technique for concurrency, but not the only one.
iooi 13y ago
> So we got concurrent units of work executed at the same time, I fail to see how that is not parallel. I thought that by definition this is impossible under the GIL. Not completely sure, but would love to know. I have written thousands of lines using gevent and eventlet but have only achieved peaks of 10 Mb/s (on servers that have at least 100), and I'm sure that truly concurrent languages could fully take advantage of that throughput -- currently in the process of migrating from Python.
- rdtsc 13y ago
  It is hard to tell but if you mostly fetch data without processing it, copying it, compute it, then gevent can certainly handle concurrent socket operations. Python's GIL won't let you execute Python code in parallel like say you start multiplying numbers in one thread and another. You won't multiply twice as many numbers because of the GIL. But for IO concurrency you should achieve parallelism (unless you have a string CPU consuming part in there as well).
- ddorian43 13y ago
  what language are you migrating to ?
- meowface 13y ago
  If you have anything that's even ever so briefly blocking gevent, then you will indeed be bottlenecked in that way. I'd be curious to see if your migration does allow better bandwidth peaks, though.