4 ms·

TIL GNU grep is not in the real world. Consider re-reading my comment. It anticipated your criticism and responded directly to it.

chaostheory 6y ago
Yes, you've basically gone over the details of why doing parallelized / concurrent performance comparisons between different programming languages is very difficult. I agree with most of what you have written. However, this doesn't change the fact that any comparison of performance between different languages, that ignores relevant concurrent and / or parallelization features, isn't very useful. This is especially true when the selling point for a few languages in this list is concurrency and / or parallization related features.
- burntsushi 6y ago
  No, what I'm saying is that it is useful. Because in a concurrent program, you'll almost certainly being using the single threaded techniques developed here. And that the fastest program will be the one that uses the fastest single threaded techniques. So it makes sense to simplify the model and just look at the thing that is going to dominate the runtime of the program. If a "dumb" thread pool is indeed what ends up being the best concurrent solution, then Rust, Go, C and C++ (and probably most of the other languages) are going to have almost no problems implementing it. It's just Not That Interesting. Now, maybe I'm wrong. Maybe there is something clever here to exploit in a concurrent program, and that some languages let you do that easier than others. I doubt it. It would be interesting if it turned out to be the case, but it would be a different benchmark. It wouldn't just be about how well a language can deal with simple data transformation tasks, but also about how it deals with tricky concurrency problems. But this doesn't seem like the kind of problem that needs such things. It complicates the model. Like honestly, just saying that any benchmark that ignores parallelism "isn't very useful" is just totally ridiculous.
  - chaostheory 6y ago
    > No, what I'm saying is that it is useful. Because in a concurrent program, you'll almost certainly being using the single threaded techniques developed here. And that the fastest program will be the one that uses the fastest single threaded techniques. You have a point. Maybe I am wrong in saying that it's near useless. > If a "dumb" thread pool is indeed what ends up being the best concurrent solution, then Rust, Go, C and C++ (and probably most of the other languages) are going to have almost no problems implementing it. It's just Not That Interesting. Your argument makes sense if every language used the same form of concurrency or parallelism. Unfortunately, that isn't the case. Wouldn't the type of concurrency or parallelism affect performance? > Now, maybe I'm wrong. Maybe there is something clever here to exploit in a concurrent program, and that some languages let you do that easier than others. but concurrency / parallelism IS much easier in some languages vs others (did I misunderstand what you wrote?).
    - burntsushi 6y ago
      > Wouldn't the type of concurrency or parallelism affect performance? That's what I'm saying: almost certainly not IF the best concurrent solution to this problem is to chunk of the work and feed it off to a bog standard thread pool. Pretty much every language---even C---has tools that make this pretty easy to do. And I don't see any reason why a dumb thread pool wouldn't be the best answer to this if parallelism were allowed. Like, the fact that Go has goroutines is just not going to help with anything for this kind of work-load. Goroutines help in more complex concurrent situations. > but concurrency / parallelism IS much easier in some languages vs others (did I misunderstand what you wrote?). When the extent of the parallelism is a dumb thread pool with no synchronization between them, then it's pretty easy to do that in just about any language.
      - chaostheory 6y ago
        > That's what I'm saying: almost certainly not IF the best concurrent solution to this problem is to chunk of the work and feed it off to a bog standard thread pool. That's the thing. This is not the case for all programming languages, even for some of the ones included this performance test. Python has a GIL. Distributing work via threads isn't going to do much unless it's something I/O intensive. Consequently, testing performance via a programming language's available methods to distribute work does matter. Special features like Goroutines also matter. > When the extent of the parallelism is a dumb thread pool with no synchronization between them, then it's pretty easy to do that in just about any language. The specific implementations of work distribution still matters for performance doesn't it? Different implementations yield varying levels of performance right?

chaostheory 6y ago

Yes, you've basically gone over the details of why doing parallelized / concurrent performance comparisons between different programming languages is very difficult. I agree with most of what you have written. However, this doesn't change the fact that any comparison of performance between different languages, that ignores relevant concurrent and / or parallelization features, isn't very useful. This is especially true when the selling point for a few languages in this list is concurrency and / or parallization related features.

burntsushi 6y ago
No, what I'm saying is that it is useful. Because in a concurrent program, you'll almost certainly being using the single threaded techniques developed here. And that the fastest program will be the one that uses the fastest single threaded techniques. So it makes sense to simplify the model and just look at the thing that is going to dominate the runtime of the program. If a "dumb" thread pool is indeed what ends up being the best concurrent solution, then Rust, Go, C and C++ (and probably most of the other languages) are going to have almost no problems implementing it. It's just Not That Interesting. Now, maybe I'm wrong. Maybe there is something clever here to exploit in a concurrent program, and that some languages let you do that easier than others. I doubt it. It would be interesting if it turned out to be the case, but it would be a different benchmark. It wouldn't just be about how well a language can deal with simple data transformation tasks, but also about how it deals with tricky concurrency problems. But this doesn't seem like the kind of problem that needs such things. It complicates the model. Like honestly, just saying that any benchmark that ignores parallelism "isn't very useful" is just totally ridiculous.
- chaostheory 6y ago
  > No, what I'm saying is that it is useful. Because in a concurrent program, you'll almost certainly being using the single threaded techniques developed here. And that the fastest program will be the one that uses the fastest single threaded techniques. You have a point. Maybe I am wrong in saying that it's near useless. > If a "dumb" thread pool is indeed what ends up being the best concurrent solution, then Rust, Go, C and C++ (and probably most of the other languages) are going to have almost no problems implementing it. It's just Not That Interesting. Your argument makes sense if every language used the same form of concurrency or parallelism. Unfortunately, that isn't the case. Wouldn't the type of concurrency or parallelism affect performance? > Now, maybe I'm wrong. Maybe there is something clever here to exploit in a concurrent program, and that some languages let you do that easier than others. but concurrency / parallelism IS much easier in some languages vs others (did I misunderstand what you wrote?).
  - burntsushi 6y ago
    > Wouldn't the type of concurrency or parallelism affect performance? That's what I'm saying: almost certainly not IF the best concurrent solution to this problem is to chunk of the work and feed it off to a bog standard thread pool. Pretty much every language---even C---has tools that make this pretty easy to do. And I don't see any reason why a dumb thread pool wouldn't be the best answer to this if parallelism were allowed. Like, the fact that Go has goroutines is just not going to help with anything for this kind of work-load. Goroutines help in more complex concurrent situations. > but concurrency / parallelism IS much easier in some languages vs others (did I misunderstand what you wrote?). When the extent of the parallelism is a dumb thread pool with no synchronization between them, then it's pretty easy to do that in just about any language.
    - chaostheory 6y ago
      > That's what I'm saying: almost certainly not IF the best concurrent solution to this problem is to chunk of the work and feed it off to a bog standard thread pool. That's the thing. This is not the case for all programming languages, even for some of the ones included this performance test. Python has a GIL. Distributing work via threads isn't going to do much unless it's something I/O intensive. Consequently, testing performance via a programming language's available methods to distribute work does matter. Special features like Goroutines also matter. > When the extent of the parallelism is a dumb thread pool with no synchronization between them, then it's pretty easy to do that in just about any language. The specific implementations of work distribution still matters for performance doesn't it? Different implementations yield varying levels of performance right?