5 ms·

I can't imagine a better setup for a language flame war :). I really like debating languages, so I hope it doesn't go that direction. One of the standard cavea

I can't imagine a better setup for a language flame war :). I really like debating languages, so I hope it doesn't go that direction. One of the standard caveats with this particular benchmark game with respect to Go is idiomatic optimizations are prohibited. To use the btree example, Go's memory management is low latency and non-moving, so allocations are expensive--any Go programmer writing a performance-sensitive btree implementation would pre-allocate the nodes in a single allocation--an absolutely idiomatic and trivial optimization--but the benchmark game requires that the nodes are allocated one at a time. In other words, the C# version is idiomatic, but the Go version is expressly contrived to be slower--not a very useful comparison. Mad respect for .Net though; it's really impressive, I like the direction it's going, I'm glad it exists, etc.

abledon 5y ago
reminds me of the 'im tired of being a hipster' post on frontpage recently.... "go learn 'unhip' tech/languages and live your life"
- na85 5y ago
  This one? https://news.ycombinator.com/item?id=28274485 https://news.ycombinator.com/item?id=28274485
igouy 5y ago
> absolutely idiomatic and trivial optimization Which is not accepted for the C# programs either. sync.Pool is accepted — https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/binarytrees-go-6.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > the Go version is expressly contrived to be slower The requirements were contrived in April 2008. afaict Go initial release was March 2012.
- throwaway894345 5y ago
  > Which is not accepted for the C# programs either. Because C# doesn't benefit from this kind of optimization. Its GC is generational, which means that it has very fast allocations at the expense of high latency. In most applications, lower latency is more important than slower allocations (not least of all because these batch-allocating optimizations are nearly trivial), but these benchmarks don't reflect that at all. > The requirements were contrived in April 2008. afaict Go initial release was March 2012. Contrived = "the rules artificially prohibit idiomatic optimizations". It doesn't require that the maintainers have a prejudice against Go (although as you point out, the maintainers have had a decade to revisit their rules).
  - igouy 5y ago
    > Because C# doesn't benefit… C# does provide a memory pool implementation.
    - throwaway894345 5y ago
      So what about C, Rust, and C++? They’re all allowed to use bespoke pools and custom allocators. The fastest Rust implementation imports an allocator crate. No doubt you can lawyer the rules to make sure Go still appears slow, but in reality this benchmark doesn’t tell you anything about the language’s general performance because while Go’s allocator is slower, idiomatic Go allocates much less frequently than other languages, but this benchmark prohibits idiomatic optimizations. Moreover, there isn’t a benchmark that shows gc latency, which is the flip side of the allocator coin. So if you really want to die on the hill of “worthless but well-lawyered benchmark rules” be my guest.
      - igouy 5y ago
        What-about-ism. https://golang.org/doc/faq#garbage_collection https://golang.org/doc/faq#garbage_collection > They’re all allowed to use bespoke pools and custom allocators. No. They are allowed a library memory pool. As-it-says: 'Please don't implement your own custom "arena" or "memory pool" or "free list" - they will not be accepted.' > … while Go’s allocator is slower… So that tiny tiny program shows it's slower because it's slower.
        throwaway894345 5y ago
        > What-about-ism. Not sure what you're referring to here. > No. They are allowed a library memory pool. Yes, this is a contrived rule. In reality, a Go developer would write the extra ~dozen lines (all of the heavy lifting done by the builtin slice implementation) and call it a day. > So that tiny tiny program shows it's slower because it's slower. Tautology. It's slower because the contrived rules preclude idiomatic optimizations. My point is that these contrived benchmarks don't indicate anything about the relative performance of these languages, but you keep responding with some variation of "but Go is slower in these benchmarks!" which everyone already agrees with. So unless you're going to actually address the point, I don't see the point in continuing on. It feels like you're hell-bent on using this thread for your personal programming language holy war, which is disinteresting to me (see again my first post) and against the site rules.
lalaithion 5y ago
The point of the btree example is to test how good programming languages are at allocating tree-like structures that can't be preplanned. It's a valid argument that this is a rare real-world requirement, but it's not contrived to be slower.
- throwaway894345 5y ago
  > The point of the btree example is to test how good programming languages are at allocating tree-like structures that can't be preplanned. Forcing allocations for every node isn't justified by a desire to demonstrate dynamically sized binary trees. A naive dynamically-sized tree would just keep a list of node buffers and allocate a new node buffer every time the previous one fills up (perhaps with subsequent buffers doubling in size). The benchmark is, by all appearances, contrived to be slower.
  - igouy 5y ago
    > … a desire to demonstrate dynamically sized binary trees… Cart before horse — the binary trees are justified by a desire to demonstrate memory allocation. http://hboehm.info/gc/gc_bench/ http://hboehm.info/gc/gc_bench/
    - throwaway894345 5y ago
      1. That’s plainly not the case here since other languages are allowed to use custom allocators 2. Why use a binary tree benchmark in the first place if you’re going to limit the implementation to certain naive implementations (and again, only for one language)? Why not just measure allocations outright or at least call the benchmark “allocator performance”? 3. Showing allocation performance doesn’t help anyone understand the actual performance of the language, which is transparently what everyone uses these benchmarks for. If they wanted a general idea for language performance they would allow trivial, idiomatic optimizations. A benchmark that shows allocation performance is worthless, and a suite of benchmarks that includes a benchmark for allocation performance but not GC latency is worse than worthless: it’s misleading because latency is the more important concern and it’s what these bump allocators trade in order to get their fast allocation performance.
_ph_ 5y ago
That is one reason I don't consider the language benchmark to be really relevant: a lot of benchmarks are taited by the exact rules of the competition.
- throwaway894345 5y ago
  And these rules are particularly bizarre. Rust, C, and C++ are all allowed to use custom allocators while Java and C# have GCs which are optimized for this particular micro benchmark but not for real world applications (although I hear Java’s GCs are making good headway on latency lately, and clearly all of the GCs are suitable for general application development). So it’s really just Go which is forbidden from an idiomatic optimization as far as I can tell.
  - igouy 5y ago
    > … allowed to use custom allocators… No. They are allowed a library memory pool. As-it-says: 'Please don't implement your own custom "arena" or "memory pool" or "free list" - they will not be accepted.'
    - throwaway894345 5y ago
      See "contrived rules" conversations elsewhere.
  - Jweb_Guru 5y ago
    Go optimizes ridiculously insanely for latency because it's driven by Hacker News articles about GC latency, not because it's better for "real world applications." Its atrocious throughput is entirely a consequence of that decision and is one of very the few useful things that the benchmark games do actually demonstrate. Java's unwillingness to provide an allocator with such an extreme tradoeff has proven a pretty good idea, and now they are able to provide only moderately worse latency than Go for "real world applications" with far better throughput.

abledon 5y ago

reminds me of the 'im tired of being a hipster' post on frontpage recently.... "go learn 'unhip' tech/languages and live your life"

na85 5y ago
This one? https://news.ycombinator.com/item?id=28274485 https://news.ycombinator.com/item?id=28274485

igouy 5y ago

> absolutely idiomatic and trivial optimization Which is not accepted for the C# programs either. sync.Pool is accepted — https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/binarytrees-go-6.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > the Go version is expressly contrived to be slower The requirements were contrived in April 2008. afaict Go initial release was March 2012.

throwaway894345 5y ago
> Which is not accepted for the C# programs either. Because C# doesn't benefit from this kind of optimization. Its GC is generational, which means that it has very fast allocations at the expense of high latency. In most applications, lower latency is more important than slower allocations (not least of all because these batch-allocating optimizations are nearly trivial), but these benchmarks don't reflect that at all. > The requirements were contrived in April 2008. afaict Go initial release was March 2012. Contrived = "the rules artificially prohibit idiomatic optimizations". It doesn't require that the maintainers have a prejudice against Go (although as you point out, the maintainers have had a decade to revisit their rules).
- igouy 5y ago
  > Because C# doesn't benefit… C# does provide a memory pool implementation.
  - throwaway894345 5y ago
    So what about C, Rust, and C++? They’re all allowed to use bespoke pools and custom allocators. The fastest Rust implementation imports an allocator crate. No doubt you can lawyer the rules to make sure Go still appears slow, but in reality this benchmark doesn’t tell you anything about the language’s general performance because while Go’s allocator is slower, idiomatic Go allocates much less frequently than other languages, but this benchmark prohibits idiomatic optimizations. Moreover, there isn’t a benchmark that shows gc latency, which is the flip side of the allocator coin. So if you really want to die on the hill of “worthless but well-lawyered benchmark rules” be my guest.
    - igouy 5y ago
      What-about-ism. https://golang.org/doc/faq#garbage_collection https://golang.org/doc/faq#garbage_collection > They’re all allowed to use bespoke pools and custom allocators. No. They are allowed a library memory pool. As-it-says: 'Please don't implement your own custom "arena" or "memory pool" or "free list" - they will not be accepted.' > … while Go’s allocator is slower… So that tiny tiny program shows it's slower because it's slower.
      - throwaway894345 5y ago
        > What-about-ism. Not sure what you're referring to here. > No. They are allowed a library memory pool. Yes, this is a contrived rule. In reality, a Go developer would write the extra ~dozen lines (all of the heavy lifting done by the builtin slice implementation) and call it a day. > So that tiny tiny program shows it's slower because it's slower. Tautology. It's slower because the contrived rules preclude idiomatic optimizations. My point is that these contrived benchmarks don't indicate anything about the relative performance of these languages, but you keep responding with some variation of "but Go is slower in these benchmarks!" which everyone already agrees with. So unless you're going to actually address the point, I don't see the point in continuing on. It feels like you're hell-bent on using this thread for your personal programming language holy war, which is disinteresting to me (see again my first post) and against the site rules.

lalaithion 5y ago

The point of the btree example is to test how good programming languages are at allocating tree-like structures that can't be preplanned. It's a valid argument that this is a rare real-world requirement, but it's not contrived to be slower.

throwaway894345 5y ago
> The point of the btree example is to test how good programming languages are at allocating tree-like structures that can't be preplanned. Forcing allocations for every node isn't justified by a desire to demonstrate dynamically sized binary trees. A naive dynamically-sized tree would just keep a list of node buffers and allocate a new node buffer every time the previous one fills up (perhaps with subsequent buffers doubling in size). The benchmark is, by all appearances, contrived to be slower.
- igouy 5y ago
  > … a desire to demonstrate dynamically sized binary trees… Cart before horse — the binary trees are justified by a desire to demonstrate memory allocation. http://hboehm.info/gc/gc_bench/ http://hboehm.info/gc/gc_bench/
  - throwaway894345 5y ago
    1. That’s plainly not the case here since other languages are allowed to use custom allocators 2. Why use a binary tree benchmark in the first place if you’re going to limit the implementation to certain naive implementations (and again, only for one language)? Why not just measure allocations outright or at least call the benchmark “allocator performance”? 3. Showing allocation performance doesn’t help anyone understand the actual performance of the language, which is transparently what everyone uses these benchmarks for. If they wanted a general idea for language performance they would allow trivial, idiomatic optimizations. A benchmark that shows allocation performance is worthless, and a suite of benchmarks that includes a benchmark for allocation performance but not GC latency is worse than worthless: it’s misleading because latency is the more important concern and it’s what these bump allocators trade in order to get their fast allocation performance.

_ph_ 5y ago

That is one reason I don't consider the language benchmark to be really relevant: a lot of benchmarks are taited by the exact rules of the competition.

throwaway894345 5y ago
And these rules are particularly bizarre. Rust, C, and C++ are all allowed to use custom allocators while Java and C# have GCs which are optimized for this particular micro benchmark but not for real world applications (although I hear Java’s GCs are making good headway on latency lately, and clearly all of the GCs are suitable for general application development). So it’s really just Go which is forbidden from an idiomatic optimization as far as I can tell.
- igouy 5y ago
  > … allowed to use custom allocators… No. They are allowed a library memory pool. As-it-says: 'Please don't implement your own custom "arena" or "memory pool" or "free list" - they will not be accepted.'
  - throwaway894345 5y ago
    See "contrived rules" conversations elsewhere.
- Jweb_Guru 5y ago
  Go optimizes ridiculously insanely for latency because it's driven by Hacker News articles about GC latency, not because it's better for "real world applications." Its atrocious throughput is entirely a consequence of that decision and is one of very the few useful things that the benchmark games do actually demonstrate. Java's unwillingness to provide an allocator with such an extreme tradoeff has proven a pretty good idea, and now they are able to provide only moderately worse latency than Go for "real world applications" with far better throughput.