16 ms·

Go vs. Crystal Performance

197 points by open-source-ux 6y ago

piinbinary 6y ago
I wonder how the GC pause times and throughput compare. My understanding is that Go sacrifices some compute performance for better GC pause times.
- dgb23 6y ago
  > better GC pause times What does that mean exactly / in this case? My assumption is that "better" means more predictable. Or does it mean straight up fewer/shorter?
  - recursivecaveat 6y ago
    Go sacrifices basically every other GC metric to minimize average pause times. Sortof a 'if you can't succeed, redefine success' mentality if you will.
    - sharpy 6y ago
      That's not necessarily a bad thing for writing web services. I used to spend a fair bit of time tuning JVM to achieve acceptable tail latencies for Java services, not to mention once in a while, they would need to be tuned again... Not so with Go.
      - apta 6y ago
        Java now has two low latency collectors: ZGC and Shenandoah.
    - jashmatthews 6y ago
      The Go GC and the JVM GCs are built within very, very different constraints. Since most short-lived allocations in Go are stack-allocated, the cost of adding a read-barrier and branching on every heap access is more expensive than the potential gain from bump-pointer allocation.
  - mr_aks 6y ago
    In Go's case, it means that GC pauses are shorter because it uses a concurrent mark and sweep algorithm.
  - gen220 6y ago
    Rick Hudson gave this excellent keynote on some of the "recent" (as of 2018) improvements and guiding principles for Go's GC, if you're interested in the nitty-gritty. https://blog.golang.org/ismmkeynote https://blog.golang.org/ismmkeynote He outlines the SLOs for the garbage collector, with the motivation that this SLO impacts the SLOs/time budget of every application built with Go. They key metrics are "share of CPU used for GC", heap sizes, latency and frequency of pauses, and scaling favorably with go-routine allocations.
cfors 6y ago
Sounds like another win for languages with an LLVM backend. Definitely excited to see this language grow, especially as it has generics already. Does anybody use Crystal professionally here? Any thoughts so far if so?
- kyrra 6y ago
  I'd be interested to see these same benchmarks with the Go LLVM frontend. https://go.googlesource.com/gollvm/ https://go.googlesource.com/gollvm/
- mauricio 6y ago
  We run it in production. Our apps are mostly Ruby but we have been rewriting services in Crystal. Originally we were attracted by the speed and type checks, but one surprising benefit has also been the reduced memory consumption. It's difficult to compare directly, but in some cases it cosumes 10x less memory and performs around 20-35x better. Even I/O bound services are sped up since we can take advantage of Crystal's concurrent fibers (rumored to come in Ruby 3.0). The main downside has been the somewhat frequent deprecations of methods and changes to the standard lib. But it's mostly due to the preparation to launch 1.0.
  - Exuma 6y ago
    Is there any web framework for it, like rails? That's really the main reason I use ruby is because as a single developer it's amazingly fast to set up new apps (admin panels, reporting panels, sales funnels, etc). The out-of-the-box functionality of rails makes this super painless. I'd LOVE to use something faster but I don't really want to have to custom write a lot of things (CSRF, sessions, cookies, blah blah). Also, how do you handle jobs with Crystal? Sidekiq being the common one for ruby.
    - obahareth 6y ago
      There are a couple but I think the most Rails-like are Amber or Lucky: https://amberframework.org/ https://amberframework.org/ https://luckyframework.org/ https://luckyframework.org/ There's also sidekiq.cr from the same person who made Sidekiq for Ruby: https://github.com/mperham/sidekiq.cr https://github.com/mperham/sidekiq.cr
ed25519FUUU 6y ago
It’s always interesting when people compare Go to other languages they always use file size of binary, as if that’s something that anyone in modern professional engineering considers. Another good example would be to statically cross-compile a non-contrived program into ARM64, 32-bit linux, and Darwin without needing google! And if you think you have your binary statically linked, go test it in the “scratch” docker image. You may be surprised at how difficult it really is.
- pjmlp 6y ago
  Sure we do, I have spent the last week trying to upload 10 MB files in a quite slow uplink, and yes in Europe, suburban area of a German town. First reason why people uninstall apps on mobile devices is app size.
  - sk0g 6y ago
    Are mobile apps the target market for Go? CI/CD takes care of upload for the dev side at least. It would be handy to have a compile time flag indicating the desired file size optimisation level still, with tradeoffs being ease of debugging, and performance.
    - pjmlp 6y ago
      Given that gomobile exists, maybe. CI/CD is something almost unknown outside HN bubble. Also that was just two examples, here are another two, cost of production for USB Armoury keys running Go bare metal, or download costs for WebAssembly modules written in Tiny Go.
      - sk0g 6y ago
        Are you sure re: CI/CD? I was looking around the job market recently, and every role was interested in my experience in it, since they were using it too. To be honest, that could be self-selecting, as my resume likely attracts companies similar to ones I have worked at. I still do hear about microservices, CI/CD a whole lot in job descriptions etc, but whether they are already practicing it (well), is another question... I was going to suggest TinyGo! I mainly work on a backend system, so I guess we have different priorities and needs. The embedded work I have done has all been C/ ASM, though it could be fun to revisit with Go.
dcu 6y ago
the HTTP benchmark is not fair since the crystal implementation is setting the content type explicitly while the Go implementation is auto detecting it.
- Thaxll 6y ago
  Indeed the benchmark was dismissed on Reddit coupe of days ago: https://www.reddit.com/r/programming/comments/h0knmi/go_vs_crystal_performance/ https://www.reddit.com/r/programming/comments/h0knmi/go_vs_c...
  - dgb23 6y ago
    The repost[0] in r/golang shows similar critiques. This perf comparison is naive on many levels. - binary size (static vs. dynamic linking) - recursion is not idiomatic in go, it is assumed that you write imperative for loops. - mathematical functions like the Fibonacci sequence are a rather atypical computation use-case for a Go program (I don't know about Crystal). Tree/graph traversal/mutations would be a more fitting test. Or generally something that is composed of dynamically growing and shrinking slices and maps. - http test apparently cannot be reproduced, some get better results for Go, some for Crystal. - http tests w/o involving some parsing/marshaling/serialization or something along those lines aren't that useful. You usually want to either read or send some JSON string or similar. [0] https://www.reddit.com/r/golang/comments/h0kogq/go_vs_crystal_performance/ https://www.reddit.com/r/golang/comments/h0kogq/go_vs_crysta...
    - yxhuvud 6y ago
      Recursion is really not the idiomatic way to solve most things in Crystal either, not that it make the test relevant.
  - coldtea 6y ago
    That said, "was dismissed on Reddit", is one of the less convincing arguments
    - 6y ago
samuell 6y ago
Anybody knows the status of light-weight threads, channels and automatic multiplexing ("mapping") of light-weight threads on operating system threads in Crystal? (This has been promised to come in Crystal (as opposed to pretty much any other language than Go), but has always seemed to be "yet to come").
- mauricio 6y ago
  It's available https://crystal-lang.org/reference/guides/concurrency.html https://crystal-lang.org/reference/guides/concurrency.html. Fibers (like goroutines) are scheduled by Crystal and map to system threads. The work on parallelism is available as a compile-time flag, but not yet GA: https://crystal-lang.org/2019/09/06/parallelism-in-crystal.html https://crystal-lang.org/2019/09/06/parallelism-in-crystal.h...
  - kodablah 6y ago
    One wonders, once it is GA, if there is value in a Go transpiler. There is a lot of useful software in Go land that would be immediately useful for Crystal devs, and Go itself is not too complicated to map to another language with the same features (a few of the runtime features will have to be emulated).
    - unixhero 6y ago
      How would this be achieved? Could you expand a little on this idea?
      - hootbootscoot 6y ago
        an IR interface, lol
      - kodablah 6y ago
        Literally convert Go code to Crystal code. I have not looked into Crystal enough to confirm it's features are a superset of Go's. For example, I saw that Kotlin w/ the advent of their coroutines, had most of Go's features so I wrote a transpiler[0] that got pretty far (can run all of this [0]). I abandoned the project because I have abandoned the JVM. 0 - https://github.com/cretz/go2k https://github.com/cretz/go2k 1 - https://github.com/cretz/go2k/tree/master/compiler/src/test/go https://github.com/cretz/go2k/tree/master/compiler/src/test/...
danielsokil 6y ago
Here Is a Gist where I compare Crystal, Go, OCaml including compile time. https://gist.github.com/s0kil/155b78580d1b68768a6c601a66f8e29b https://gist.github.com/s0kil/155b78580d1b68768a6c601a66f8e2...
- dleslie 6y ago
  Why did you combine compile time and execution time?
  - tobyhinloopen 6y ago
    Development I suppose
  - danielsokil 6y ago
    The idea is to have a balance of compilation + runtime performance.
- vips7L 6y ago
  I did this in Java on Jdk 8 and it took 6 seconds.
- mratsim 6y ago
  So Crystal is trying to beat C++?
deleted 6y ago
[deleted]
systemvoltage 6y ago
I am not an expert at this but I want to inquire: - Everytime there is any kind of benchmark between 2 or more languages, there is always a caveat - "Not a fair comparison" or the algorithm isn't right. What, then, is a good way to compare languages? After all, they are not apples-oranges comparison. They are a tool to get things done. It is like I am comparing 2 different brands of hammers. Sure the grip is different, the shape of the head is different, but if we are testing a particular aspect of nailing - say nails/sec, then it is worth investigating which one is better. One hammer takes a lot of expertise to use it and yet another one is easy to use. So, then, write those pros and cons down, it doesn't make the comparison a worthless activity which is the cliche response to benchmarks - developer time matters and not the language. What is a good way to compare 2 languages. Say Python vs. C? Are there standard implementations of algorithms (e.g. Mandelbrot) or something like that we can use and definitively compare the speed? Shouldn't we have standards around these benchmarks? Some kind of a ISO implementation of an algorithm reviewed by experts that can be used for benchmarking? For majority of programmers, the speed doesn't quite matter. But that discussion is off the table and orthogonal. Sure for most things, I personally pick up Python and it is the fastest way to develop for me. But there are many reasons why we should compare languages and these concerns are shouldn't stop us from evaluating objectively.
- icholy 6y ago
  The problem is that solutions optimised for benchmarks are usually non-idiomatic. If the solution doesn't represent "real code", then is there any real value? An interesting benchmarking approach would be to take solutions to the same problem from many different programmers at different skill levels and gather metrics from them. I think that would give a much more interesting view into a language's performance characteristics.
  - igouy 6y ago
    > If the solution doesn't represent "real code", then is there any real value? When "idiomatic" performs so poorly that the software is unfit for purpose, does that represent "real code" ? > … solutions to the same problem from many different programmers at different skill levels… How could we know programs weren't written to make some language look bad?
sbmthakur 6y ago
Has anyone compared Crystal and Rust on similar lines?
- unixhero 6y ago
  It would be cool to see. Crystal-lang isn't finished yet tho. Maybe it's a little too early for that?
- tracker1 6y ago
  Did a rust build with the following... fn fibonacci(n: u32) -> u32 { match n { 0 => 1, 1 => 1, _ => fibonacci(n - 1) + fibonacci(n - 2), } } fn main() { println!("{}", fibonacci(47)); } size: 2.67kb time: 7.391s I think rust wins :-D --- edit: not sure if I used valgrind/massif right, but the largest values in the output file. mem_heap_B=1816 mem_heap_extra_B=32 I think this means it stayed under 2k ram?
  - tracker1 6y ago
    For the web server... 3.6mb file using hyper, not the leanest but one of the more popular options. extern crate hyper; use hyper::Server; use hyper::server::Request; use hyper::server::Response; fn hello(req: Request, res: Response) { let response = format!("Hello from {}!", req.remote_addr.ip()); res.send(response.as_bytes()).unwrap(); } fn main() { Server::http("127.0.0.1:3000").unwrap() .handle(hello).unwrap(); } Output from wrk -t8 -c400 -d60s http://localhost:3000/ http://localhost:3000/ Running 1m test @ http://localhost:3000/ 8 threads and 400 connections Thread Stats Avg Stdev Max +/- Stdev Latency 173.31us 88.10us 0.98ms 65.18% Req/Sec 35.06k 13.46k 45.48k 74.99% 8383194 requests in 1.00m, 775.50MB read Socket errors: connect 0, read 0, write 229, timeout 0 Requests/sec: 139487.00 Transfer/sec: 12.90MB According to top, rust_http is using about 2.8mb ram, 630% cpu, and wrk is using 700k and 370% cpu. Though I'm on completely different hardware, but the thing definitely screams... For reference: AMD Ryzen R9 3950X, 64GB DDR4@3200(CL18) under WSL2-Ubuntu (Windows 10 insiders)
  - 6y ago
awb 6y ago
This is a pretty basic benchmark, but there are more robust ones for Crystal and others here: https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/ Under Filters you can select the languages / frameworks that are of interest to you. For example Go seems to have several implementations that beat Crystal: https://www.techempower.com/benchmarks/#section=data-r19&hw=ph&test=fortune&l=zdjvnj-1r https://www.techempower.com/benchmarks/#section=data-r19&hw=... Having programmed in both though, I far prefer the language / idea of Crystal for innovation. But it is hard to beat the performance and explicit nature of Go for production applications.
- throwaway894345 6y ago
  The non-recursive Go fibonacci implementation computes in ~160ms and uses only 1.5mb on my MacBookPro (and I can get the binary size down to 850K by stripping symbols and using the println builtin instead of fmt.Println). This microbenchmark seems to tell us more about function call overhead than it does about typical application performance. Further, while I don't doubt that Crystal binary sizes are smaller in general, I don't think we can get much information about binary sizes from these micro-toy-sized programs. For example, the Go version shaved off a full mb by using the `println` builtin rather than `fmt.Println`, probably because the former pulls in more of the runtime than the latter. These toy benchmarks are very sensitive to these kinds of things, and the size of the runtime is going to be a trivial portion of the size of any real application anyway.
strictfp 6y ago
Consider using wrk2 in order to avoid coordinated omission.
euph0ria 6y ago
Why doesn't Crystal use more than 100% CPU like Go does if the server has 8 cores? Why didn't both languages max out the cores? Is it because the params to WRK was within what one core could handle in Crystal?
- WatchDog 6y ago
  It seems crystal is single threaded by default. Although you can configure it to use more threads. https://crystal-lang.org/2019/09/06/parallelism-in-crystal.html https://crystal-lang.org/2019/09/06/parallelism-in-crystal.h...
  - euph0ria 6y ago
    Yeah, but in a benchmark it seems strange that he doesn't restrict both to use just one CPU or unlimited.. kind of compares apples/oranges.
jjtheblunt 6y ago
Might binary size comparisons be misleading, unless the Crystal binary is statically linked, as presumably the Go binary is?
mikece 6y ago
Tangent: would technical writers please stop using the phrasing “is X times smaller” when “is one Xth the size” is more accurate? I had to read the reference to “Crystal’s binary size is 5 times smaller than Go’s” twice to realize it was 0.2x and not 5x.
- jiofih 6y ago
  I don’t get it. “5x smaller” and “0.2x the size” are the same. Can you explain then what do you expect to be the meaning of “two times smaller”?
  - mikece 6y ago
    Clarity of language for one thing, but also that "times" is indicative of multiplication so "five times" anything has to be bigger[1]. One fifth is far clearer as there's no way to interpret that other than being a part of the size of the reference, ergo, smaller. Is it impossible to understand? No, but it's poor style for technical writing. [1] I'm ignoring the case of decimal math since the reference in this case is always the whole integer 1.
    - zanellato19 6y ago
      I think this takes the cake for the most pedantic HN comment I have ever read.
    - jiofih 6y ago
      I don’t know under which rock you’ve been hiding but “x times faster/smaller” has been in common use for decades, including technical writing. Everyone understands it as 1/X, there is absolutely no confusion.
- omaranto 6y ago
  If a thing has size x it has smallness 1/x (so the bigger the thing, the less smallness it has). When you say one thing is 5x smaller than another, it means the smallness of the first is 5 times the smallness of the second. :P
pier25 6y ago
Now compare Crystal with Nim.
- nobleach 6y ago
  I want to say this: https://framework.embarklabs.io/news/2019/11/18/nim-vs-crystal-part-1-performance-interoperability/index.html https://framework.embarklabs.io/news/2019/11/18/nim-vs-cryst... was published on HN at one point. Regardless, it's a decent rundown.
  - pier25 6y ago
    Oh wow I expected Nim to be at least as fast as Crystal.
    - mratsim 6y ago
      Nim JSON parser is not optimized, it was mostly written for maintenance (for example it allocates a Table per node in the json file) I.e. the difference in speed here is a difference in elbow grease. I have yet to come into an optimization problem where you cannot reach the speed you can achieve in C in Nim.
totalperspectiv 6y ago
Completely anecdotal: - In my small benchmarks, Crystal is very fast - In benchmarks not my own, Crystal is very fast - Crystal has a very nice batteries included stdlib. Benchmarks that aren't mine: http://lh3.github.io/2020/05/17/fast-high-level-programming-languages http://lh3.github.io/2020/05/17/fast-high-level-programming-... The only other language that is high level in its ballpark is D.
- mratsim 6y ago
  Benchmarking strings is often benchmarking memory allocation speed or the GC. If you manipulate strings day in day out in your workflow, the first thing you do is remove all those allocations and for what is left you use a memory pool. The speedup could easily go into the 20x in any language. It's actually one of the area where Python or JS can beat statically typed languages, they focused a lot into optimizing strings while statically typed languages often leave that as an exercise to the reader.
  - throwaway894345 6y ago
    > It's actually one of the area where Python or JS can beat statically typed languages, they focused a lot into optimizing strings while statically typed languages often leave that as an exercise to the reader. I think this has more to do with writing the string handling functions in heavily-optimized C rather than in Python/JS than it does with memory tricks, but I'll happily accept correction.
    - echlebek 6y ago
      Almost certainly. Iteration is around 40x slower in Python than in a compiled language, last I checked.
    - mratsim 6y ago
      Yes low-level C + retaining memory around in the GC. The second part avoids many malloc/free that may be hidden in destructors in low-level languages.
  - 6y ago
decafbad 6y ago
I suggest the word femtobenchmark for this kind of work.
seabass 6y ago
It seems strange that Crystal would not utilize more than 100% of the CPU in the HTTP benchmark given that it ran on an 8-core machine and with fibers should be able to split the load across multiple cores. What explanation is there for that? Also, it's impressive that while using one third of the CPU Crystal outperformed the Go HTTP server on throughput.
- yxhuvud 6y ago
  Because currently Crystal is singlethreaded. You only get multithreading if you compile with -Dpreview_mt . I suppose it will be on by default at some later point.
dzonga 6y ago
I always find these benchmarks superficial. unless you're doing some serious number crunching, it doesn't make sense. people need to start bench marking languages on speed of development, developer ergonomics, error reporting, ease to deploy etc.
zelly 6y ago
Benchmarks don't matter. It's 2020. We use Stock Price Driven Development now. The main reason to use Swift is Apple created it. The main reason to use Go is it's the first two letters of Google. The main reason to use Python is Google hired GvR in the 2000s and created momentum. The main reason to use Kotlin is Google announced at I/O 2017 its support in Android Studio. Get with the times gramps.
- 7532yahoogmail 6y ago
  Wow. No. Engineering school was/is supposed to help us (you) move beyond cynical judgements to thinking.
  - crimsonalucard5 6y ago
    I could care less if a statement was positive or negative what is important is if the statement has a realistic possibility of being true. If anything Engineering school should have taught you how to logically observe situations rather than viewing things through an optimistic or cynical lens.
- hombre_fatal 6y ago
  The only time I've ever heard anyone assert this is in posts like this where you lambast everyone but yourself for being an idiot.
karmakaze 6y ago
I've taken Crystal out for a spin with a number of popular and less popular frameworks. Even for my small test application, edit/compile/run times were slow. Only the thinnest frameworks like Kemal seem tolerable to me. I really do hope that the compiler gets faster and the larger frameworks figure out how to compile faster. These are the benchmarks that matter to me before I'd make a recommendation.
- Trasmatta 6y ago
  My understanding is that the compiler performance issues in Crystal are sort of inherent to the language design and making it much faster will be very hard. I believe it's related to Crystal's type inference -- it has to traverse every code branch to ensure type safety. I believe explicitly typing everything is supposed to help, but I'm not sure by how much (especially if you have dependencies that aren't doing that).
  - entha_saava 6y ago
    maybe LLVM as well..
gigatexal 6y ago
Interesting. But more important is how productive can I be and how expansive is the ecosystem? What’s the developer experience like? Is the community welcoming?
poorman 6y ago
I'm still waiting for someone to write an LLVM backend for Go.
- hootbootscoot 6y ago
  gollvm? vs gogcc. that's why i said "an IR interface" when someone asked about transpiling crystal to go and vice-versa.
skyzyx 6y ago
Without having done any of my own research, I’m initially skeptical of Crystal’s binary size. I initially saw something similar with Swift, but that’s just because the runtime is external to the binary.

I wonder how the GC pause times and throughput compare. My understanding is that Go sacrifices some compute performance for better GC pause times.

dgb23 6y ago
> better GC pause times What does that mean exactly / in this case? My assumption is that "better" means more predictable. Or does it mean straight up fewer/shorter?
- recursivecaveat 6y ago
  Go sacrifices basically every other GC metric to minimize average pause times. Sortof a 'if you can't succeed, redefine success' mentality if you will.
  - sharpy 6y ago
    That's not necessarily a bad thing for writing web services. I used to spend a fair bit of time tuning JVM to achieve acceptable tail latencies for Java services, not to mention once in a while, they would need to be tuned again... Not so with Go.
    - apta 6y ago
      Java now has two low latency collectors: ZGC and Shenandoah.
  - jashmatthews 6y ago
    The Go GC and the JVM GCs are built within very, very different constraints. Since most short-lived allocations in Go are stack-allocated, the cost of adding a read-barrier and branching on every heap access is more expensive than the potential gain from bump-pointer allocation.
- mr_aks 6y ago
  In Go's case, it means that GC pauses are shorter because it uses a concurrent mark and sweep algorithm.
- gen220 6y ago
  Rick Hudson gave this excellent keynote on some of the "recent" (as of 2018) improvements and guiding principles for Go's GC, if you're interested in the nitty-gritty. https://blog.golang.org/ismmkeynote https://blog.golang.org/ismmkeynote He outlines the SLOs for the garbage collector, with the motivation that this SLO impacts the SLOs/time budget of every application built with Go. They key metrics are "share of CPU used for GC", heap sizes, latency and frequency of pauses, and scaling favorably with go-routine allocations.

cfors 6y ago

Sounds like another win for languages with an LLVM backend. Definitely excited to see this language grow, especially as it has generics already. Does anybody use Crystal professionally here? Any thoughts so far if so?

kyrra 6y ago
I'd be interested to see these same benchmarks with the Go LLVM frontend. https://go.googlesource.com/gollvm/ https://go.googlesource.com/gollvm/
mauricio 6y ago
We run it in production. Our apps are mostly Ruby but we have been rewriting services in Crystal. Originally we were attracted by the speed and type checks, but one surprising benefit has also been the reduced memory consumption. It's difficult to compare directly, but in some cases it cosumes 10x less memory and performs around 20-35x better. Even I/O bound services are sped up since we can take advantage of Crystal's concurrent fibers (rumored to come in Ruby 3.0). The main downside has been the somewhat frequent deprecations of methods and changes to the standard lib. But it's mostly due to the preparation to launch 1.0.
- Exuma 6y ago
  Is there any web framework for it, like rails? That's really the main reason I use ruby is because as a single developer it's amazingly fast to set up new apps (admin panels, reporting panels, sales funnels, etc). The out-of-the-box functionality of rails makes this super painless. I'd LOVE to use something faster but I don't really want to have to custom write a lot of things (CSRF, sessions, cookies, blah blah). Also, how do you handle jobs with Crystal? Sidekiq being the common one for ruby.
  - obahareth 6y ago
    There are a couple but I think the most Rails-like are Amber or Lucky: https://amberframework.org/ https://amberframework.org/ https://luckyframework.org/ https://luckyframework.org/ There's also sidekiq.cr from the same person who made Sidekiq for Ruby: https://github.com/mperham/sidekiq.cr https://github.com/mperham/sidekiq.cr

ed25519FUUU 6y ago

It’s always interesting when people compare Go to other languages they always use file size of binary, as if that’s something that anyone in modern professional engineering considers. Another good example would be to statically cross-compile a non-contrived program into ARM64, 32-bit linux, and Darwin without needing google! And if you think you have your binary statically linked, go test it in the “scratch” docker image. You may be surprised at how difficult it really is.

pjmlp 6y ago
Sure we do, I have spent the last week trying to upload 10 MB files in a quite slow uplink, and yes in Europe, suburban area of a German town. First reason why people uninstall apps on mobile devices is app size.
- sk0g 6y ago
  Are mobile apps the target market for Go? CI/CD takes care of upload for the dev side at least. It would be handy to have a compile time flag indicating the desired file size optimisation level still, with tradeoffs being ease of debugging, and performance.
  - pjmlp 6y ago
    Given that gomobile exists, maybe. CI/CD is something almost unknown outside HN bubble. Also that was just two examples, here are another two, cost of production for USB Armoury keys running Go bare metal, or download costs for WebAssembly modules written in Tiny Go.
    - sk0g 6y ago
      Are you sure re: CI/CD? I was looking around the job market recently, and every role was interested in my experience in it, since they were using it too. To be honest, that could be self-selecting, as my resume likely attracts companies similar to ones I have worked at. I still do hear about microservices, CI/CD a whole lot in job descriptions etc, but whether they are already practicing it (well), is another question... I was going to suggest TinyGo! I mainly work on a backend system, so I guess we have different priorities and needs. The embedded work I have done has all been C/ ASM, though it could be fun to revisit with Go.

dcu 6y ago

the HTTP benchmark is not fair since the crystal implementation is setting the content type explicitly while the Go implementation is auto detecting it.

Thaxll 6y ago
Indeed the benchmark was dismissed on Reddit coupe of days ago: https://www.reddit.com/r/programming/comments/h0knmi/go_vs_crystal_performance/ https://www.reddit.com/r/programming/comments/h0knmi/go_vs_c...
- dgb23 6y ago
  The repost[0] in r/golang shows similar critiques. This perf comparison is naive on many levels. - binary size (static vs. dynamic linking) - recursion is not idiomatic in go, it is assumed that you write imperative for loops. - mathematical functions like the Fibonacci sequence are a rather atypical computation use-case for a Go program (I don't know about Crystal). Tree/graph traversal/mutations would be a more fitting test. Or generally something that is composed of dynamically growing and shrinking slices and maps. - http test apparently cannot be reproduced, some get better results for Go, some for Crystal. - http tests w/o involving some parsing/marshaling/serialization or something along those lines aren't that useful. You usually want to either read or send some JSON string or similar. [0] https://www.reddit.com/r/golang/comments/h0kogq/go_vs_crystal_performance/ https://www.reddit.com/r/golang/comments/h0kogq/go_vs_crysta...
  - yxhuvud 6y ago
    Recursion is really not the idiomatic way to solve most things in Crystal either, not that it make the test relevant.
- coldtea 6y ago
  That said, "was dismissed on Reddit", is one of the less convincing arguments
  - 6y ago

samuell 6y ago

Anybody knows the status of light-weight threads, channels and automatic multiplexing ("mapping") of light-weight threads on operating system threads in Crystal? (This has been promised to come in Crystal (as opposed to pretty much any other language than Go), but has always seemed to be "yet to come").

mauricio 6y ago
It's available https://crystal-lang.org/reference/guides/concurrency.html https://crystal-lang.org/reference/guides/concurrency.html. Fibers (like goroutines) are scheduled by Crystal and map to system threads. The work on parallelism is available as a compile-time flag, but not yet GA: https://crystal-lang.org/2019/09/06/parallelism-in-crystal.html https://crystal-lang.org/2019/09/06/parallelism-in-crystal.h...
- kodablah 6y ago
  One wonders, once it is GA, if there is value in a Go transpiler. There is a lot of useful software in Go land that would be immediately useful for Crystal devs, and Go itself is not too complicated to map to another language with the same features (a few of the runtime features will have to be emulated).
  - unixhero 6y ago
    How would this be achieved? Could you expand a little on this idea?
    - hootbootscoot 6y ago
      an IR interface, lol
    - kodablah 6y ago
      Literally convert Go code to Crystal code. I have not looked into Crystal enough to confirm it's features are a superset of Go's. For example, I saw that Kotlin w/ the advent of their coroutines, had most of Go's features so I wrote a transpiler[0] that got pretty far (can run all of this [0]). I abandoned the project because I have abandoned the JVM. 0 - https://github.com/cretz/go2k https://github.com/cretz/go2k 1 - https://github.com/cretz/go2k/tree/master/compiler/src/test/go https://github.com/cretz/go2k/tree/master/compiler/src/test/...

danielsokil 6y ago

Here Is a Gist where I compare Crystal, Go, OCaml including compile time. https://gist.github.com/s0kil/155b78580d1b68768a6c601a66f8e29b https://gist.github.com/s0kil/155b78580d1b68768a6c601a66f8e2...

dleslie 6y ago
Why did you combine compile time and execution time?
- tobyhinloopen 6y ago
  Development I suppose
- danielsokil 6y ago
  The idea is to have a balance of compilation + runtime performance.
vips7L 6y ago
I did this in Java on Jdk 8 and it took 6 seconds.
mratsim 6y ago
So Crystal is trying to beat C++?

deleted 6y ago

[deleted]

systemvoltage 6y ago

I am not an expert at this but I want to inquire: - Everytime there is any kind of benchmark between 2 or more languages, there is always a caveat - "Not a fair comparison" or the algorithm isn't right. What, then, is a good way to compare languages? After all, they are not apples-oranges comparison. They are a tool to get things done. It is like I am comparing 2 different brands of hammers. Sure the grip is different, the shape of the head is different, but if we are testing a particular aspect of nailing - say nails/sec, then it is worth investigating which one is better. One hammer takes a lot of expertise to use it and yet another one is easy to use. So, then, write those pros and cons down, it doesn't make the comparison a worthless activity which is the cliche response to benchmarks - developer time matters and not the language. What is a good way to compare 2 languages. Say Python vs. C? Are there standard implementations of algorithms (e.g. Mandelbrot) or something like that we can use and definitively compare the speed? Shouldn't we have standards around these benchmarks? Some kind of a ISO implementation of an algorithm reviewed by experts that can be used for benchmarking? For majority of programmers, the speed doesn't quite matter. But that discussion is off the table and orthogonal. Sure for most things, I personally pick up Python and it is the fastest way to develop for me. But there are many reasons why we should compare languages and these concerns are shouldn't stop us from evaluating objectively.

icholy 6y ago
The problem is that solutions optimised for benchmarks are usually non-idiomatic. If the solution doesn't represent "real code", then is there any real value? An interesting benchmarking approach would be to take solutions to the same problem from many different programmers at different skill levels and gather metrics from them. I think that would give a much more interesting view into a language's performance characteristics.
- igouy 6y ago
  > If the solution doesn't represent "real code", then is there any real value? When "idiomatic" performs so poorly that the software is unfit for purpose, does that represent "real code" ? > … solutions to the same problem from many different programmers at different skill levels… How could we know programs weren't written to make some language look bad?

sbmthakur 6y ago

Has anyone compared Crystal and Rust on similar lines?

unixhero 6y ago
It would be cool to see. Crystal-lang isn't finished yet tho. Maybe it's a little too early for that?
tracker1 6y ago
Did a rust build with the following... fn fibonacci(n: u32) -> u32 { match n { 0 => 1, 1 => 1, _ => fibonacci(n - 1) + fibonacci(n - 2), } } fn main() { println!("{}", fibonacci(47)); } size: 2.67kb time: 7.391s I think rust wins :-D --- edit: not sure if I used valgrind/massif right, but the largest values in the output file. mem_heap_B=1816 mem_heap_extra_B=32 I think this means it stayed under 2k ram?
- tracker1 6y ago
  For the web server... 3.6mb file using hyper, not the leanest but one of the more popular options. extern crate hyper; use hyper::Server; use hyper::server::Request; use hyper::server::Response; fn hello(req: Request, res: Response) { let response = format!("Hello from {}!", req.remote_addr.ip()); res.send(response.as_bytes()).unwrap(); } fn main() { Server::http("127.0.0.1:3000").unwrap() .handle(hello).unwrap(); } Output from wrk -t8 -c400 -d60s http://localhost:3000/ http://localhost:3000/ Running 1m test @ http://localhost:3000/ 8 threads and 400 connections Thread Stats Avg Stdev Max +/- Stdev Latency 173.31us 88.10us 0.98ms 65.18% Req/Sec 35.06k 13.46k 45.48k 74.99% 8383194 requests in 1.00m, 775.50MB read Socket errors: connect 0, read 0, write 229, timeout 0 Requests/sec: 139487.00 Transfer/sec: 12.90MB According to top, rust_http is using about 2.8mb ram, 630% cpu, and wrk is using 700k and 370% cpu. Though I'm on completely different hardware, but the thing definitely screams... For reference: AMD Ryzen R9 3950X, 64GB DDR4@3200(CL18) under WSL2-Ubuntu (Windows 10 insiders)
- 6y ago

awb 6y ago

This is a pretty basic benchmark, but there are more robust ones for Crystal and others here: https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/ Under Filters you can select the languages / frameworks that are of interest to you. For example Go seems to have several implementations that beat Crystal: https://www.techempower.com/benchmarks/#section=data-r19&hw=ph&test=fortune&l=zdjvnj-1r https://www.techempower.com/benchmarks/#section=data-r19&hw=... Having programmed in both though, I far prefer the language / idea of Crystal for innovation. But it is hard to beat the performance and explicit nature of Go for production applications.

throwaway894345 6y ago
The non-recursive Go fibonacci implementation computes in ~160ms and uses only 1.5mb on my MacBookPro (and I can get the binary size down to 850K by stripping symbols and using the println builtin instead of fmt.Println). This microbenchmark seems to tell us more about function call overhead than it does about typical application performance. Further, while I don't doubt that Crystal binary sizes are smaller in general, I don't think we can get much information about binary sizes from these micro-toy-sized programs. For example, the Go version shaved off a full mb by using the `println` builtin rather than `fmt.Println`, probably because the former pulls in more of the runtime than the latter. These toy benchmarks are very sensitive to these kinds of things, and the size of the runtime is going to be a trivial portion of the size of any real application anyway.

strictfp 6y ago

Consider using wrk2 in order to avoid coordinated omission.

euph0ria 6y ago

Why doesn't Crystal use more than 100% CPU like Go does if the server has 8 cores? Why didn't both languages max out the cores? Is it because the params to WRK was within what one core could handle in Crystal?

WatchDog 6y ago
It seems crystal is single threaded by default. Although you can configure it to use more threads. https://crystal-lang.org/2019/09/06/parallelism-in-crystal.html https://crystal-lang.org/2019/09/06/parallelism-in-crystal.h...
- euph0ria 6y ago
  Yeah, but in a benchmark it seems strange that he doesn't restrict both to use just one CPU or unlimited.. kind of compares apples/oranges.

jjtheblunt 6y ago

Might binary size comparisons be misleading, unless the Crystal binary is statically linked, as presumably the Go binary is?

mikece 6y ago

Tangent: would technical writers please stop using the phrasing “is X times smaller” when “is one Xth the size” is more accurate? I had to read the reference to “Crystal’s binary size is 5 times smaller than Go’s” twice to realize it was 0.2x and not 5x.

jiofih 6y ago
I don’t get it. “5x smaller” and “0.2x the size” are the same. Can you explain then what do you expect to be the meaning of “two times smaller”?
- mikece 6y ago
  Clarity of language for one thing, but also that "times" is indicative of multiplication so "five times" anything has to be bigger[1]. One fifth is far clearer as there's no way to interpret that other than being a part of the size of the reference, ergo, smaller. Is it impossible to understand? No, but it's poor style for technical writing. [1] I'm ignoring the case of decimal math since the reference in this case is always the whole integer 1.
  - zanellato19 6y ago
    I think this takes the cake for the most pedantic HN comment I have ever read.
  - jiofih 6y ago
    I don’t know under which rock you’ve been hiding but “x times faster/smaller” has been in common use for decades, including technical writing. Everyone understands it as 1/X, there is absolutely no confusion.
omaranto 6y ago
If a thing has size x it has smallness 1/x (so the bigger the thing, the less smallness it has). When you say one thing is 5x smaller than another, it means the smallness of the first is 5 times the smallness of the second. :P

pier25 6y ago

Now compare Crystal with Nim.

nobleach 6y ago
I want to say this: https://framework.embarklabs.io/news/2019/11/18/nim-vs-crystal-part-1-performance-interoperability/index.html https://framework.embarklabs.io/news/2019/11/18/nim-vs-cryst... was published on HN at one point. Regardless, it's a decent rundown.
- pier25 6y ago
  Oh wow I expected Nim to be at least as fast as Crystal.
  - mratsim 6y ago
    Nim JSON parser is not optimized, it was mostly written for maintenance (for example it allocates a Table per node in the json file) I.e. the difference in speed here is a difference in elbow grease. I have yet to come into an optimization problem where you cannot reach the speed you can achieve in C in Nim.

totalperspectiv 6y ago

Completely anecdotal: - In my small benchmarks, Crystal is very fast - In benchmarks not my own, Crystal is very fast - Crystal has a very nice batteries included stdlib. Benchmarks that aren't mine: http://lh3.github.io/2020/05/17/fast-high-level-programming-languages http://lh3.github.io/2020/05/17/fast-high-level-programming-... The only other language that is high level in its ballpark is D.

mratsim 6y ago
Benchmarking strings is often benchmarking memory allocation speed or the GC. If you manipulate strings day in day out in your workflow, the first thing you do is remove all those allocations and for what is left you use a memory pool. The speedup could easily go into the 20x in any language. It's actually one of the area where Python or JS can beat statically typed languages, they focused a lot into optimizing strings while statically typed languages often leave that as an exercise to the reader.
- throwaway894345 6y ago
  > It's actually one of the area where Python or JS can beat statically typed languages, they focused a lot into optimizing strings while statically typed languages often leave that as an exercise to the reader. I think this has more to do with writing the string handling functions in heavily-optimized C rather than in Python/JS than it does with memory tricks, but I'll happily accept correction.
  - echlebek 6y ago
    Almost certainly. Iteration is around 40x slower in Python than in a compiled language, last I checked.
  - mratsim 6y ago
    Yes low-level C + retaining memory around in the GC. The second part avoids many malloc/free that may be hidden in destructors in low-level languages.
- 6y ago

decafbad 6y ago

I suggest the word femtobenchmark for this kind of work.

seabass 6y ago

It seems strange that Crystal would not utilize more than 100% of the CPU in the HTTP benchmark given that it ran on an 8-core machine and with fibers should be able to split the load across multiple cores. What explanation is there for that? Also, it's impressive that while using one third of the CPU Crystal outperformed the Go HTTP server on throughput.

yxhuvud 6y ago
Because currently Crystal is singlethreaded. You only get multithreading if you compile with -Dpreview_mt . I suppose it will be on by default at some later point.

dzonga 6y ago

I always find these benchmarks superficial. unless you're doing some serious number crunching, it doesn't make sense. people need to start bench marking languages on speed of development, developer ergonomics, error reporting, ease to deploy etc.

zelly 6y ago

Benchmarks don't matter. It's 2020. We use Stock Price Driven Development now. The main reason to use Swift is Apple created it. The main reason to use Go is it's the first two letters of Google. The main reason to use Python is Google hired GvR in the 2000s and created momentum. The main reason to use Kotlin is Google announced at I/O 2017 its support in Android Studio. Get with the times gramps.

7532yahoogmail 6y ago
Wow. No. Engineering school was/is supposed to help us (you) move beyond cynical judgements to thinking.
- crimsonalucard5 6y ago
  I could care less if a statement was positive or negative what is important is if the statement has a realistic possibility of being true. If anything Engineering school should have taught you how to logically observe situations rather than viewing things through an optimistic or cynical lens.
hombre_fatal 6y ago
The only time I've ever heard anyone assert this is in posts like this where you lambast everyone but yourself for being an idiot.

karmakaze 6y ago

I've taken Crystal out for a spin with a number of popular and less popular frameworks. Even for my small test application, edit/compile/run times were slow. Only the thinnest frameworks like Kemal seem tolerable to me. I really do hope that the compiler gets faster and the larger frameworks figure out how to compile faster. These are the benchmarks that matter to me before I'd make a recommendation.

Trasmatta 6y ago
My understanding is that the compiler performance issues in Crystal are sort of inherent to the language design and making it much faster will be very hard. I believe it's related to Crystal's type inference -- it has to traverse every code branch to ensure type safety. I believe explicitly typing everything is supposed to help, but I'm not sure by how much (especially if you have dependencies that aren't doing that).
- entha_saava 6y ago
  maybe LLVM as well..

gigatexal 6y ago

Interesting. But more important is how productive can I be and how expansive is the ecosystem? What’s the developer experience like? Is the community welcoming?

poorman 6y ago

I'm still waiting for someone to write an LLVM backend for Go.

hootbootscoot 6y ago
gollvm? vs gogcc. that's why i said "an IR interface" when someone asked about transpiling crystal to go and vice-versa.

skyzyx 6y ago

Without having done any of my own research, I’m initially skeptical of Crystal’s binary size. I initially saw something similar with Swift, but that’s just because the runtime is external to the binary.