9 ms·

Programming language comparison by reimplementing the same transit data app

201 points by simonpure 4y ago

joshlemer 4y ago
The experience report on Scala I find pretty cathartic. It really is absolutely ridiculous the fetishization of extremely complex FP and type-level hacking that goes on in the ecosystem, to the point where, in the case of the author's hello world web server snippet, it's just so complex and laden with concepts you need to know that are unrelated to the problem you're trying to solve it could be mistaken for a parody or exaggeration, but it's actually the library recommended most often to beginners sadly.
- losvedir 4y ago
  It's a shame, too, because until that point (when I was still just following the "getting started" docs and doing the first half of the app, which loads a CSV and parses it and stuff), I was actually really enjoying it! The type-magic style of development is so different from what I'm used to, I have periodic crises of conscience where I wonder if I've been doing programming wrong all these years, or if it's an example of a community barking up the wrong tree. This thinking is also what prompts me to look into APL/J/K every so often. The last time I really tried to get into hardcore FP programming was several years ago with Haskell, but even then I don't recall Yesod (the web framework I tried out, akin to http4s here) being quite so overwhelming.
  - chowells 4y ago
    And Yesod is one of the worst options in Haskell, at least as far as making me wonder why it's doing so much magic. It's very much trying to build a fully integrated framework instead of being composable Haskell. (To be fair, the underlying parts written to implement it are composable Haskell, and re-used by a lot of other web server projects.)
    - bradrn 4y ago
      I think ‘worst’ is very subjective here. It certainly does aim to be an all-encompassing ‘framework’ — but this is hardly unusual amongst web libraries (not just for Haskell!), and I feel Yesod gets the job done pretty well. Of course, Haskell has many alternatives if you don’t like Yesod: amongst other libraries, there’s Servant [0], snap [1], scotty [2], and the lower-level wai [3] and warp [4] if you feel the need for that kind of control. [0] https://hackage.haskell.org/package/servant https://hackage.haskell.org/package/servant [1] https://hackage.haskell.org/package/snap https://hackage.haskell.org/package/snap [2] https://hackage.haskell.org/package/scotty https://hackage.haskell.org/package/scotty [3] https://hackage.haskell.org/package/wai https://hackage.haskell.org/package/wai [4] https://hackage.haskell.org/package/warp https://hackage.haskell.org/package/warp
  - hoosieree 4y ago
    I dove into the "type driven development" style with Elm (and a little Haskell) a few years back. In my opinion the pure functional style was more important than the type system. Maybe I just didn't try hard enough to think in terms of types, who knows. But I have to admit - not having to worry about run-time errors is quite nice. Shortly after Elm, I discovered the array languages via J, and felt like I finally found a style of programming that fit my brain properly. Immutable by default, rank-polymorphic, and extremely powerful. Terseness is a feature - when you can write the same program in 1/100 of the code (not an exaggeration), you can explore alternative approaches quickly and find bugs more easily. On the other hand, there's no compiler or fancy type system to find bugs for you, so terseness is also kind of necessary. Today I'd recommend array language newbies to try BQN or K first. BQN has some features that other array languages surprisingly lack (closures, modules), and only a few idiosyncratic design choices (unlike J which is very weird). K is pragmatic (it has dictionaries and tables!) and is ascii-based, but it has an almost Forth-like minimalism. For someone used to 'import xyz' style programming it's a jarring transition.
  - KingOfCoders 4y ago
    And from founding a startup on Scala (Lift + Play), grow over some years and sell, I most dread compiler errors with the type magic. If something doesn't compile because of types, the error messages (at least some years back) were so confusing, it took quite some time to fix problems.
  - valenterry 4y ago
    The http4s library was a bad choice in your case and it's at least partly their fault. They should really make it clear from the very beginning that you MUST understand certain PFP concepts and datatypes to effectively use the library. It is a NOT a library that you should start out with when you learn Scala, unless you have a mentor. I think for you, cask was the right choice and it's good that you found it.
  - phibz 4y ago
    > I have periodic crises of conscience where I wonder if I've been doing programming wrong all these years, or if it's an example of a community barking up the wrong tree. When I first learned of FP about 20 years ago I thought it was a terrible idea. But I've convinced myself I have been proven wrong time and time again. I still think sometime it's complexity for its own sake. Eg. why is it so hard to define a monad? But I enjoyed parts of Scala 2 and later Rust. There are good ideas in FP that lead to programs that are easier to reason about and manage complexity. Languages are borrowing these good ideas and I'm happier for it, algebraic data types, pattern matching and destructuring, result/either, immutability, etc
  - shirogane86x 4y ago
    as someone who did (and still does) a lot of Haskell, and actually loves it, seeing http4s reminded me less of Yesod and more of Servant: really high flexibility, advanced type level magic, and a fairly steep learning curve - especially so if you're not accustomed to advanced typed functional programming.
- karmakaze 4y ago
  This has been my experience as well, both individually and on a team. Everyone says it's fine as long as you show restraint and stay within bounds you collectively find comfortable/manageable. In practice, we all reach a point of understanding/comfort that doesn't stay with you on reading later, even by the author.
- kenhwang 4y ago
  The Scala 3 requirement really does cripple the choices available. The Scala 3 transition does feel a bit like the Python 3 transition where major libraries and frameworks are really dragging their feet on transitioning because of the high level of effort for very little obvious improvement. Scala 2 is still getting regular updates so there's no rush to use 3 just yet, I'd recommend picking the framework first and using the Scala version they support. That makes the development experience much better with access to much easier to use frameworks like play or finatra.
  - vvillena 4y ago
    In the long term it will be an improvement. The one thing dragging adoption down is the removal of the old macro system, which depended completely on the implementation details of the old compiler. That said, I agree that Scala 2 is still very much supported, so there's nothing wrong with picking it.
- solidninja 4y ago
  I do wonder where the recommendation to use http4s for beginners came from. http4s is a very capable library (and if you care much about composition it is excellent), but I wouldn't describe the documentation as beginner friendly. A slightly better starting point for scala 3 + type-safe server building is tapir e.g. https://github.com/softwaremill/tapir/blob/master/examples3/src/main/scala/sttp/tapir/examples3/ErrorUnionTypesHttp4sServer.scala https://github.com/softwaremill/tapir/blob/master/examples3/... . With that, you get a declarative definition of your endpoints (+ error types, auth, etc.) that you can use for both servers and clients, which comes very handy when writing integration tests of course. > absolutely ridiculous the fetishization of extremely complex FP and type-level hacking that goes on in the ecosystem An alternative way to look at it is that there is a lot of essential domain complexity that gets encoded via the type system to let the compiler do the hard work. That "extremely complex FP" does not arrive out of nowhere - I really recommend at least skimming through the slides from rossabaker, the http4s designer, that motivate where the core type signature comes from https://rossabaker.github.io/boston-http4s/#2 https://rossabaker.github.io/boston-http4s/#2 I suppose one of the "features" that I like about the (typelevel) community is that the approach of "worse is better" is not taken, and a lot of effort is expended to make things correct, modular and orthogonal. This has the drawback of increased upfront complexity, that anecdotally pays off the moment your compiler does not error and the program runs as intended.
  - klabb3 4y ago
    > http4s is a very capable library (and if you care much about composition it is excellent), but I wouldn't describe the documentation as beginner friendly. A slightly better starting point for scala 3 + type-safe server building is tapir I am not a scala user but this sounds like a serious violation of composability (ironically), for something so ubiquitous as http. I don't really mind advanced features being available for power users, or later optimizations, but I really dislike being forced to make a mutually exclusive decision between simple and performant, early in the project lifecycle. I think Rust unfortunately often falls in this category as well with async and multiple http libraries, and even the inofficial doctrine of deferring ownership decisions for later. In Go there's basically one way to do things, and the few more advanced things you can do can be opted into later. I assume this is a quality inherited from C, and it's extremely pragmatic and creates a composable ecosystem.
- vvillena 4y ago
  I agree with the sentiment that the more FP-heavy parts of the Scala ecosystem are not suited to hello-world code _at all_. In this particular case, http4s is at its core a low-level library. The author was right to go for Play first. But it turns out there's an alrernative. Tapir is emerging as that "pure FP, but approachable" HTTP framework. It allows developers to work at a higher level, turning the HTTP backend into an implementation detail. Simply define the application using Tapir, then choose the better backend for the use case.
losvedir 4y ago
Oh, hey, didn't expect to see this here. Thanks for submitting! I tried a few days ago but didn't really get any traction.[0] Since I submitted it, though, I posted it to /r/rust and got a lot of feedback. At the time, rust and dotnet were comparable and at the top of the list, with ~10k req/sec. Now rust is far and away the most performant at ~20k req/sec! I also was able to improve Go's performance 30% or so. Still, I want to let the other communities chip in and see if I can improve them. I was actually just in the midst of exploring how to improve Elixir's results. I'm finding I can almost double my requests per second from simply switching the JSON encoder from Jason to Jiffy. That sort of surprised me since Jason is the de-facto standard, and I thought was super fast. [0] https://news.ycombinator.com/item?id=33291604 https://news.ycombinator.com/item?id=33291604
- di4na 4y ago
  Are you on OTP 25 ? With the JIT, in theory Jason is faster than Jiffy
  - di4na 4y ago
    Another thing to add, do you log things in the other languages ? If not, what you are probably benchmarking there is how fast your console is in receiving the text (with a lock). Also use a release, it will help some things here, especially on "short" benchmark, and make sure that the JIT is run.
    - losvedir 4y ago
      Yep, Elixir 1.14, OTP 25. I set the log level to warn so Phoenix isn't logging anything that I see. Do you think that still impacts performance? I thought Logger compiled out log calls below the level threshold. Good call on running as a release! I'll try that next. edit: Ah well, good thoughts but didn't pan out. I updated the logger config to use `compile_time_purge_matching` just to make sure there wouldn't be any logging impact, and I ran the app as a release, but didn't really make any difference in the numbers that I saw.
- bhaney 4y ago
jph 4y ago
Excellent writeup thank you. And I love that you wrote "BurntSushi is a national treasure". I hope he sees your writeup because it's true. (BurntSushi is Andrew Gallant, creator of ripgrep and other excellent Rust crates, and writes code that is especially good IMHO for learning how to write real-world Rust libraries and programs.)
zikohh 4y ago
I would've expected a higher requests per sec for 10VU for Go. Throughout the comparisons Go was just right behind rust but here it it was off by a high magnitude.
andreygrehov 4y ago
According to the results, Go and Rust pretty much beat every other language. While Rust is the clear winner, it has a steep learning curve. When it comes to Rust vs Go, I always choose Go, because it still beats all the other languages and at the same time an extremely simple language to learn / read / write. This triplet has a greater weight in my books.
- dopu 4y ago
  I've been learning Go recently and it's been such a pleasure, particularly for 2D gamedev (with ebitengine.org). I don't think I've gone from reading the spec to being able to read others' programs fluently as quickly as I was able to with Go. Plus it's so fast! Compilation times had me thinking it was interpreted when I first started playing around with it.
- awestroke 4y ago
  I always choose rust, because the tooling and language feel thought through and it's easy to write once you learn it.
- super_flanker 4y ago
  Would never choose Go unless I'm paid for me. Rust is so much better to work with, pattern match and algebraic data type just makes the programming so much fun.
throwawaymaths 4y ago
What I would love to see is someone injecting an programming error into something like this and seeing how long it takes for someone experienced in the language to debug it.
makapuf 4y ago
Interesting results. Do you think it would be interesting /possible to try sqlite as a pure in memory db using :memory db file?
metaltyphoon 4y ago
You could have used minimal apis in C# and gotten a better throughput
jamie_ca 4y ago
For a similar project, someone put together a spec for a basic clone of Medium split into a frontend design and backend API, and now has over 100 different components where you can pair any frontend with any backend (plus a few fullstack implementations). https://github.com/gothinkster/realworld https://github.com/gothinkster/realworld Not as performance-focused with benchmarks, but a good point of comparison for various languages and frameworks implementing common behavior.
bottlepalm 4y ago
How do you think Node.js would fare here, similar to Deno?
- christophilus 4y ago
  I’d like to see that, too. I’ve been able to get Node to match Go performance (roughly) when building SQLite-based applications by having all of the writes happen on one thread (passing the work to it via Node IPC), and doing all reads on read-only instances in the workers. Node really craps the bed when you start pulling in libraries, though. And there are lots of performance surprises such as fetch being quite slow vs the original http client, etc.
llamaLord 4y ago
You mention Deno maybe not handling multiple cores “well”. I’m 99.9% sure Deno doesn’t handle multiple cores at all. Everything in the node ecosystem is one big asynchronous single-threaded event loop.
- junon 4y ago
  There are indeed threads in some cases. I believe Deno uses libuv, which means that certain calls (iirc the syscall for DNS resolution) are put into separate threads since there are no true non-blocking versions of them provided by the kernel.
labrador 4y ago
> As for the language, C# is... all right, I guess. It kind of reminds me of Dart That's right because Google's Dart (2010's) was heavily influenced by Microsoft's C# (2000's) which was heavily influenced by Sun's Java (1990's) I've often wondered if would have been better if the companies had worked together to produce a standard VM language
- pasc1878 4y ago
  You need to look at the history of Microsoft's version of Java to see why that did not happen.
esprehn 4y ago
This was fun to read through. I would need to profile the code, but the startup time being bad for Deno seems like maybe a combination of the code in here being unoptimized: https://github.com/denoland/deno_std/blob/0ce558fec1a1beeda3ed7710c6507b9e991829c9/encoding/csv/_parser.ts#L138 https://github.com/denoland/deno_std/blob/0ce558fec1a1beeda3... (Ex. Lots of temporaries) And usage of the readFileSync+TextDecoder API instead of readTextFile (which is also a docs issue since it suggests the first one). It seems the code loads the 100MB into memory, then converts to another 100MB of utf8, then parses with that inefficient csv parser. The rust and go versions look to be doing stream/incremental processing instead.
irq-1 4y ago
> It took me longer than I'd like to admit to figure out how to get a dang io.Reader, which is what the CSV parsing package takes. In Go you want to start with the package examples. CSV has NewReader(). I had the same experience with Java... you know what a function takes, but where do you get one of those?
sonicgear1 4y ago
Use minimal APIs in .NET and get a 2X perf boost.
Stamp01 4y ago
Maybe I inferred something that wasn't actually implied, but it sounds like the author's use of Sqlite was with a disk-backed database. I'd be curious to see how the languages compared to an in-memory only Sqlite database's times.
KingOfCoders 4y ago
What again and again I find interesting, that Rust is (slightly) faster than Go (with all the discussions about generics etc.) Still I would try a framework with Go, it seems odd to not use one when a framework is used with the other languages.
- belmont_sup 4y ago
  I like that he made choices based on value per effort. I would’ve done the same thing too. But that’s a fair take. Go’s web server ecosystem has diversified a lot too, and I’m sure his pick would’ve affected the outcome a reasonable amount if he had chosen fasthttp or gin for comparison.
craftoman 4y ago
I don't wanna be rude but comparing "languages" by using multithreaded examples VS single threaded ones is so dumb.
- dajonker 4y ago
  If you don't want to be rude just don't comment. I don't think the author intends this to be an objective comparison by any means. It's just their subjective experience trying to build some software in different languages. See also the opening paragraph of the project's readme: > This repository implements the same simple backend API in a variety of languages. It's just a personal project of mine to get a feel for the languages, and shouldn't be taken too seriously.
int_19h 4y ago
Looking at the sources, isn't this more of a comparison between different CSV parsers - e.g. C# version has one hand-rolled, while Go uses one from stdlib?
kitd 4y ago
This is a great write-up. Personal impressions like this are as good as, if not better then, numbers and benchmarks for comparing languages, especially when covering things like how easy the docs or tooling are to use. That's the side of a language that most affects devs day-to-day. Looking forward to the reports for plain Java and Nim.

201 points by simonpure 4y ago

The experience report on Scala I find pretty cathartic. It really is absolutely ridiculous the fetishization of extremely complex FP and type-level hacking that goes on in the ecosystem, to the point where, in the case of the author's hello world web server snippet, it's just so complex and laden with concepts you need to know that are unrelated to the problem you're trying to solve it could be mistaken for a parody or exaggeration, but it's actually the library recommended most often to beginners sadly.

losvedir 4y ago
It's a shame, too, because until that point (when I was still just following the "getting started" docs and doing the first half of the app, which loads a CSV and parses it and stuff), I was actually really enjoying it! The type-magic style of development is so different from what I'm used to, I have periodic crises of conscience where I wonder if I've been doing programming wrong all these years, or if it's an example of a community barking up the wrong tree. This thinking is also what prompts me to look into APL/J/K every so often. The last time I really tried to get into hardcore FP programming was several years ago with Haskell, but even then I don't recall Yesod (the web framework I tried out, akin to http4s here) being quite so overwhelming.
- chowells 4y ago
  And Yesod is one of the worst options in Haskell, at least as far as making me wonder why it's doing so much magic. It's very much trying to build a fully integrated framework instead of being composable Haskell. (To be fair, the underlying parts written to implement it are composable Haskell, and re-used by a lot of other web server projects.)
  - bradrn 4y ago
    I think ‘worst’ is very subjective here. It certainly does aim to be an all-encompassing ‘framework’ — but this is hardly unusual amongst web libraries (not just for Haskell!), and I feel Yesod gets the job done pretty well. Of course, Haskell has many alternatives if you don’t like Yesod: amongst other libraries, there’s Servant [0], snap [1], scotty [2], and the lower-level wai [3] and warp [4] if you feel the need for that kind of control. [0] https://hackage.haskell.org/package/servant https://hackage.haskell.org/package/servant [1] https://hackage.haskell.org/package/snap https://hackage.haskell.org/package/snap [2] https://hackage.haskell.org/package/scotty https://hackage.haskell.org/package/scotty [3] https://hackage.haskell.org/package/wai https://hackage.haskell.org/package/wai [4] https://hackage.haskell.org/package/warp https://hackage.haskell.org/package/warp
- hoosieree 4y ago
  I dove into the "type driven development" style with Elm (and a little Haskell) a few years back. In my opinion the pure functional style was more important than the type system. Maybe I just didn't try hard enough to think in terms of types, who knows. But I have to admit - not having to worry about run-time errors is quite nice. Shortly after Elm, I discovered the array languages via J, and felt like I finally found a style of programming that fit my brain properly. Immutable by default, rank-polymorphic, and extremely powerful. Terseness is a feature - when you can write the same program in 1/100 of the code (not an exaggeration), you can explore alternative approaches quickly and find bugs more easily. On the other hand, there's no compiler or fancy type system to find bugs for you, so terseness is also kind of necessary. Today I'd recommend array language newbies to try BQN or K first. BQN has some features that other array languages surprisingly lack (closures, modules), and only a few idiosyncratic design choices (unlike J which is very weird). K is pragmatic (it has dictionaries and tables!) and is ascii-based, but it has an almost Forth-like minimalism. For someone used to 'import xyz' style programming it's a jarring transition.
- KingOfCoders 4y ago
  And from founding a startup on Scala (Lift + Play), grow over some years and sell, I most dread compiler errors with the type magic. If something doesn't compile because of types, the error messages (at least some years back) were so confusing, it took quite some time to fix problems.
- valenterry 4y ago
  The http4s library was a bad choice in your case and it's at least partly their fault. They should really make it clear from the very beginning that you MUST understand certain PFP concepts and datatypes to effectively use the library. It is a NOT a library that you should start out with when you learn Scala, unless you have a mentor. I think for you, cask was the right choice and it's good that you found it.
- phibz 4y ago
  > I have periodic crises of conscience where I wonder if I've been doing programming wrong all these years, or if it's an example of a community barking up the wrong tree. When I first learned of FP about 20 years ago I thought it was a terrible idea. But I've convinced myself I have been proven wrong time and time again. I still think sometime it's complexity for its own sake. Eg. why is it so hard to define a monad? But I enjoyed parts of Scala 2 and later Rust. There are good ideas in FP that lead to programs that are easier to reason about and manage complexity. Languages are borrowing these good ideas and I'm happier for it, algebraic data types, pattern matching and destructuring, result/either, immutability, etc
- shirogane86x 4y ago
  as someone who did (and still does) a lot of Haskell, and actually loves it, seeing http4s reminded me less of Yesod and more of Servant: really high flexibility, advanced type level magic, and a fairly steep learning curve - especially so if you're not accustomed to advanced typed functional programming.
karmakaze 4y ago
This has been my experience as well, both individually and on a team. Everyone says it's fine as long as you show restraint and stay within bounds you collectively find comfortable/manageable. In practice, we all reach a point of understanding/comfort that doesn't stay with you on reading later, even by the author.
kenhwang 4y ago
The Scala 3 requirement really does cripple the choices available. The Scala 3 transition does feel a bit like the Python 3 transition where major libraries and frameworks are really dragging their feet on transitioning because of the high level of effort for very little obvious improvement. Scala 2 is still getting regular updates so there's no rush to use 3 just yet, I'd recommend picking the framework first and using the Scala version they support. That makes the development experience much better with access to much easier to use frameworks like play or finatra.
- vvillena 4y ago
  In the long term it will be an improvement. The one thing dragging adoption down is the removal of the old macro system, which depended completely on the implementation details of the old compiler. That said, I agree that Scala 2 is still very much supported, so there's nothing wrong with picking it.
solidninja 4y ago
I do wonder where the recommendation to use http4s for beginners came from. http4s is a very capable library (and if you care much about composition it is excellent), but I wouldn't describe the documentation as beginner friendly. A slightly better starting point for scala 3 + type-safe server building is tapir e.g. https://github.com/softwaremill/tapir/blob/master/examples3/src/main/scala/sttp/tapir/examples3/ErrorUnionTypesHttp4sServer.scala https://github.com/softwaremill/tapir/blob/master/examples3/... . With that, you get a declarative definition of your endpoints (+ error types, auth, etc.) that you can use for both servers and clients, which comes very handy when writing integration tests of course. > absolutely ridiculous the fetishization of extremely complex FP and type-level hacking that goes on in the ecosystem An alternative way to look at it is that there is a lot of essential domain complexity that gets encoded via the type system to let the compiler do the hard work. That "extremely complex FP" does not arrive out of nowhere - I really recommend at least skimming through the slides from rossabaker, the http4s designer, that motivate where the core type signature comes from https://rossabaker.github.io/boston-http4s/#2 https://rossabaker.github.io/boston-http4s/#2 I suppose one of the "features" that I like about the (typelevel) community is that the approach of "worse is better" is not taken, and a lot of effort is expended to make things correct, modular and orthogonal. This has the drawback of increased upfront complexity, that anecdotally pays off the moment your compiler does not error and the program runs as intended.
- klabb3 4y ago
  > http4s is a very capable library (and if you care much about composition it is excellent), but I wouldn't describe the documentation as beginner friendly. A slightly better starting point for scala 3 + type-safe server building is tapir I am not a scala user but this sounds like a serious violation of composability (ironically), for something so ubiquitous as http. I don't really mind advanced features being available for power users, or later optimizations, but I really dislike being forced to make a mutually exclusive decision between simple and performant, early in the project lifecycle. I think Rust unfortunately often falls in this category as well with async and multiple http libraries, and even the inofficial doctrine of deferring ownership decisions for later. In Go there's basically one way to do things, and the few more advanced things you can do can be opted into later. I assume this is a quality inherited from C, and it's extremely pragmatic and creates a composable ecosystem.
vvillena 4y ago
I agree with the sentiment that the more FP-heavy parts of the Scala ecosystem are not suited to hello-world code _at all_. In this particular case, http4s is at its core a low-level library. The author was right to go for Play first. But it turns out there's an alrernative. Tapir is emerging as that "pure FP, but approachable" HTTP framework. It allows developers to work at a higher level, turning the HTTP backend into an implementation detail. Simply define the application using Tapir, then choose the better backend for the use case.

losvedir 4y ago

Oh, hey, didn't expect to see this here. Thanks for submitting! I tried a few days ago but didn't really get any traction.[0] Since I submitted it, though, I posted it to /r/rust and got a lot of feedback. At the time, rust and dotnet were comparable and at the top of the list, with ~10k req/sec. Now rust is far and away the most performant at ~20k req/sec! I also was able to improve Go's performance 30% or so. Still, I want to let the other communities chip in and see if I can improve them. I was actually just in the midst of exploring how to improve Elixir's results. I'm finding I can almost double my requests per second from simply switching the JSON encoder from Jason to Jiffy. That sort of surprised me since Jason is the de-facto standard, and I thought was super fast. [0] https://news.ycombinator.com/item?id=33291604 https://news.ycombinator.com/item?id=33291604

di4na 4y ago
Are you on OTP 25 ? With the JIT, in theory Jason is faster than Jiffy
- di4na 4y ago
  Another thing to add, do you log things in the other languages ? If not, what you are probably benchmarking there is how fast your console is in receiving the text (with a lock). Also use a release, it will help some things here, especially on "short" benchmark, and make sure that the JIT is run.
  - losvedir 4y ago
    Yep, Elixir 1.14, OTP 25. I set the log level to warn so Phoenix isn't logging anything that I see. Do you think that still impacts performance? I thought Logger compiled out log calls below the level threshold. Good call on running as a release! I'll try that next. edit: Ah well, good thoughts but didn't pan out. I updated the logger config to use `compile_time_purge_matching` just to make sure there wouldn't be any logging impact, and I ran the app as a release, but didn't really make any difference in the numbers that I saw.
bhaney 4y ago

jph 4y ago

Excellent writeup thank you. And I love that you wrote "BurntSushi is a national treasure". I hope he sees your writeup because it's true. (BurntSushi is Andrew Gallant, creator of ripgrep and other excellent Rust crates, and writes code that is especially good IMHO for learning how to write real-world Rust libraries and programs.)

zikohh 4y ago

I would've expected a higher requests per sec for 10VU for Go. Throughout the comparisons Go was just right behind rust but here it it was off by a high magnitude.

andreygrehov 4y ago

According to the results, Go and Rust pretty much beat every other language. While Rust is the clear winner, it has a steep learning curve. When it comes to Rust vs Go, I always choose Go, because it still beats all the other languages and at the same time an extremely simple language to learn / read / write. This triplet has a greater weight in my books.

dopu 4y ago
I've been learning Go recently and it's been such a pleasure, particularly for 2D gamedev (with ebitengine.org). I don't think I've gone from reading the spec to being able to read others' programs fluently as quickly as I was able to with Go. Plus it's so fast! Compilation times had me thinking it was interpreted when I first started playing around with it.
awestroke 4y ago
I always choose rust, because the tooling and language feel thought through and it's easy to write once you learn it.
super_flanker 4y ago
Would never choose Go unless I'm paid for me. Rust is so much better to work with, pattern match and algebraic data type just makes the programming so much fun.

throwawaymaths 4y ago

What I would love to see is someone injecting an programming error into something like this and seeing how long it takes for someone experienced in the language to debug it.

makapuf 4y ago

Interesting results. Do you think it would be interesting /possible to try sqlite as a pure in memory db using :memory db file?

metaltyphoon 4y ago

You could have used minimal apis in C# and gotten a better throughput

jamie_ca 4y ago

For a similar project, someone put together a spec for a basic clone of Medium split into a frontend design and backend API, and now has over 100 different components where you can pair any frontend with any backend (plus a few fullstack implementations). https://github.com/gothinkster/realworld https://github.com/gothinkster/realworld Not as performance-focused with benchmarks, but a good point of comparison for various languages and frameworks implementing common behavior.

bottlepalm 4y ago

How do you think Node.js would fare here, similar to Deno?

christophilus 4y ago
I’d like to see that, too. I’ve been able to get Node to match Go performance (roughly) when building SQLite-based applications by having all of the writes happen on one thread (passing the work to it via Node IPC), and doing all reads on read-only instances in the workers. Node really craps the bed when you start pulling in libraries, though. And there are lots of performance surprises such as fetch being quite slow vs the original http client, etc.

llamaLord 4y ago

You mention Deno maybe not handling multiple cores “well”. I’m 99.9% sure Deno doesn’t handle multiple cores at all. Everything in the node ecosystem is one big asynchronous single-threaded event loop.

junon 4y ago
There are indeed threads in some cases. I believe Deno uses libuv, which means that certain calls (iirc the syscall for DNS resolution) are put into separate threads since there are no true non-blocking versions of them provided by the kernel.

labrador 4y ago

> As for the language, C# is... all right, I guess. It kind of reminds me of Dart That's right because Google's Dart (2010's) was heavily influenced by Microsoft's C# (2000's) which was heavily influenced by Sun's Java (1990's) I've often wondered if would have been better if the companies had worked together to produce a standard VM language

pasc1878 4y ago
You need to look at the history of Microsoft's version of Java to see why that did not happen.

esprehn 4y ago

This was fun to read through. I would need to profile the code, but the startup time being bad for Deno seems like maybe a combination of the code in here being unoptimized: https://github.com/denoland/deno_std/blob/0ce558fec1a1beeda3ed7710c6507b9e991829c9/encoding/csv/_parser.ts#L138 https://github.com/denoland/deno_std/blob/0ce558fec1a1beeda3... (Ex. Lots of temporaries) And usage of the readFileSync+TextDecoder API instead of readTextFile (which is also a docs issue since it suggests the first one). It seems the code loads the 100MB into memory, then converts to another 100MB of utf8, then parses with that inefficient csv parser. The rust and go versions look to be doing stream/incremental processing instead.

irq-1 4y ago

> It took me longer than I'd like to admit to figure out how to get a dang io.Reader, which is what the CSV parsing package takes. In Go you want to start with the package examples. CSV has NewReader(). I had the same experience with Java... you know what a function takes, but where do you get one of those?

sonicgear1 4y ago

Use minimal APIs in .NET and get a 2X perf boost.

Stamp01 4y ago

Maybe I inferred something that wasn't actually implied, but it sounds like the author's use of Sqlite was with a disk-backed database. I'd be curious to see how the languages compared to an in-memory only Sqlite database's times.

KingOfCoders 4y ago

What again and again I find interesting, that Rust is (slightly) faster than Go (with all the discussions about generics etc.) Still I would try a framework with Go, it seems odd to not use one when a framework is used with the other languages.

belmont_sup 4y ago
I like that he made choices based on value per effort. I would’ve done the same thing too. But that’s a fair take. Go’s web server ecosystem has diversified a lot too, and I’m sure his pick would’ve affected the outcome a reasonable amount if he had chosen fasthttp or gin for comparison.

craftoman 4y ago

I don't wanna be rude but comparing "languages" by using multithreaded examples VS single threaded ones is so dumb.

dajonker 4y ago
If you don't want to be rude just don't comment. I don't think the author intends this to be an objective comparison by any means. It's just their subjective experience trying to build some software in different languages. See also the opening paragraph of the project's readme: > This repository implements the same simple backend API in a variety of languages. It's just a personal project of mine to get a feel for the languages, and shouldn't be taken too seriously.

int_19h 4y ago

Looking at the sources, isn't this more of a comparison between different CSV parsers - e.g. C# version has one hand-rolled, while Go uses one from stdlib?

kitd 4y ago

This is a great write-up. Personal impressions like this are as good as, if not better then, numbers and benchmarks for comparing languages, especially when covering things like how easy the docs or tooling are to use. That's the side of a language that most affects devs day-to-day. Looking forward to the reports for plain Java and Nim.