3 ms·

What I'm missing is how an IO bound workload is really ever a good chance to showcase a language's threading and concurrency performance or featureset. When I t

What I'm missing is how an IO bound workload is really ever a good chance to showcase a language's threading and concurrency performance or featureset. When I think about a language's threading model, I think about memory sharing/message passing, synchronization and fences, signaling, context switching and scheduling, preemption etc. None of that is really important if the threads aren't doing any real work.

the_duke 10y ago
Of course it is. In many languages that don't have microthreads baked in, this is not straightforward. You either have to use OS threads, which have a way higher overhead in memory and context switching costs and are not trivial to get right. Or you use some async IO framework with callbacks or the event based OS apis directly, which is also FAR from trivial to get right. Do you really want to think about memory fences and context switching when writing concurrent code? I don't. Of course you should be aware of how it works underneath though.
- gvb 10y ago
  The problem with the article is Amdahl's law: he is speeding up the part of the code that takes a small fraction of the time and handwaving the bottleneck that takes most of the time. "Currently the bottleneck is writing to the file system..." (and then the rest of the paragraph makes no sense at all). "We cannot do writing to disk from several threads for the following reasons..." - so his threading speeds up everything except the bottleneck that takes all the time.
  - the_duke 10y ago
    I highly doubt that the disk writes were the original bottleneck. It probably is the new bottleneck, after parallelizing the data fetching and database IO. I agree that the article has some flaws, though.
dllthomas 10y ago
I don't know, if the workload is compute bound, but the threads don't do any meaningful coordination, that's awfully boring. And if the threads do a substantial amount of coordination, that's potentially interesting even if the work is predominately IO.

the_duke 10y ago

Of course it is. In many languages that don't have microthreads baked in, this is not straightforward. You either have to use OS threads, which have a way higher overhead in memory and context switching costs and are not trivial to get right. Or you use some async IO framework with callbacks or the event based OS apis directly, which is also FAR from trivial to get right. Do you really want to think about memory fences and context switching when writing concurrent code? I don't. Of course you should be aware of how it works underneath though.

gvb 10y ago
The problem with the article is Amdahl's law: he is speeding up the part of the code that takes a small fraction of the time and handwaving the bottleneck that takes most of the time. "Currently the bottleneck is writing to the file system..." (and then the rest of the paragraph makes no sense at all). "We cannot do writing to disk from several threads for the following reasons..." - so his threading speeds up everything except the bottleneck that takes all the time.
- the_duke 10y ago
  I highly doubt that the disk writes were the original bottleneck. It probably is the new bottleneck, after parallelizing the data fetching and database IO. I agree that the article has some flaws, though.

dllthomas 10y ago

I don't know, if the workload is compute bound, but the threads don't do any meaningful coordination, that's awfully boring. And if the threads do a substantial amount of coordination, that's potentially interesting even if the work is predominately IO.