9 ms·

In my current job, I was given a task to write a service which should handle billions of events eventually in the fastest possible way. The language choice was

by pimeys 10y ago

In my current job, I was given a task to write a service which should handle billions of events eventually in the fastest possible way. The language choice was given for me to decide, and I was thinking that maybe I'll do it with C++14, Scala, Go or Rust. C++ has it's quirks and I'm not really enjoying it's build tool of choice, cmake. Scala I can write fast, but scaling the app in Mesos would consume lots of memory; every task would take a huge slice of RAM because of the additional JVM. Go I just don't like as a language (personal taste) and I think the GC adds a bit too much of pausing for this app, so I gave Rust a shot. The first week was madness. I'm fairly experienced developer and the Rust compiler hit me to my fingers constantly. Usually the only way out was to write the whole part again with a different architecture. On the second week I was done with my tasks, very comfortable with the language and just enjoying the tooling. Also I was relying on tests way less because of the compiler, even less than with Scala. If it compiles, it has a big chance of working. Cargo is awesome. Let me repeat: Cargo is awesome. Also I like how I can write code with Vim again and even though for some things I need to read the Rust source, it is pretty easy to get answers if you're in trouble. In the end I wrote some integration tests with Python and I'm quite happy with them. Now I want to write more Rust.

karterk 10y ago
On the other hand, as your system gets more complex, you will start missing tried and test libraries from some of the more mature languages you have mentioned. Maybe the extra time you have to spend writing that code will compensate for the extra memory you would have spent because of the JVM. Not taking anything away from Rust - I'm just giving an alternative perspective.
- pimeys 10y ago
  That is a risk we've analyzed and are willing to take. Scala can stay on the data side of our platform, but the real time part will be either Rust or C++14 (and Python where we don't need raw speed and can handle everything with one thread). For the libraries the situation is not that bad. Something I was missing from Scala side was Kamon. It is a really awesome library I've been using (and contributing) for some time. For Rust I needed to do my own solution with threads and a bit of macro love. It works, it is using a constant amount of ram and when we get our next Rust project running, I might open source that part for everybody to use...
  - annnnd 10y ago
    As far as I can tell, Kamon is an application monitoring system - what do you monitor? I am curious as I have never felt the need to monitor apps this way (but I have never worked on HFT apps obviously).
    - pimeys 10y ago
      Counters, times and such. These are to be sent to a service like InfluxDB and monitored with a tool such as Grafana.
    - pimeys 10y ago
      Also, one thing I would love to see is the syntactic sugar for binding, like Haskell's `do` and Scala's `for`. Would be super useful for lots of things, e.g. validating requests in a web server.
- loup-vaillant 10y ago
  I bet a fair number of those tried and tested libraries expose a C interface. As far as I know, Rust have an FFI. Those libraries are only a set of bindings away. So I think we can do away with Java libraries, as well as the occasional header-only C++ template madness¹. 1: I mean, it's madness if you want to talk to it from the outside. If you stick with C++, this is "merely" flamewar material (take the Boost devs and the C++ committee on one side, and game devs like Mike Acton, Jonathan Blow, or Casey Muratori on the other side).
  - lmm 10y ago
    > I bet a fair number of those tried and tested libraries expose a C interface. Some of them do, or have C equivalents, but many of them don't. Once you get into "there's only one library that does it and it's one guy's graduate thesis" territory, often the only option is on the JVM.
    - loup-vaillant 10y ago
      > "there's only one library that does it and it's one guy's graduate thesis" Those libraries aren't tried and tested. They can be reimplemented. And in some contexts, they shouldn't be trusted as a black box.
    - jleahy 10y ago
      The last time I was in that situation, the only option was lisp. I'm still supporting a lisp environment for that one piece of code.
gizzlon 10y ago
> Go I just don't like as a language (personal taste) and I think the GC adds a bit too much of pausing for this app Not saying you have to like Go, we all have different preferences. But FYI, the gc has gotten much better in the latest releases. I don't think you would be bothered by gc pauses. (my understanding is that it' both faster and more "spread out")
- pimeys 10y ago
  We're doing real time stuff where these pauses matter. Just that we don't really need GC for anything and adding it to certain services cause more harm than it causes good.
  - gizzlon 10y ago
    Curious, how long can a pause be before it's a problem? (I do not write these kind of services)
    - jerf 10y ago
      "It depends". Some services are insensitive to pauses of several minutes (email processing; you may not want it there all the time but spikes to that level are often acceptable). Some services are sensitive to pauses in the single-digit milliseconds (for instance trying to maintain a high frame rate).
      - dllthomas 10y ago
        Two years ago I was working on a system that cared about single digit microseconds, luckily with low throughput.
        jleahy 10y ago
        Lots of people have to worry about single digit microseconds with high throughput.
lmm 10y ago
Is the JVM overhead really significant these days? You're talking about wasting what, maybe 32MB for each instance? (which can have multiple threads, you don't need one per core). Is your RAM really so limited that you'd run out of RAM before maxing out all your CPUs or I/O in a Scala version? (I am kind of interested in Rust, but not going to touch it until it gets proper HKT support - you have to duplicate so much code otherwise)
- pimeys 10y ago
  It is a lot, when we need to scale by adding more Mesos tasks. Every task launches a new JVM instance, which adds to the total memory usage. Also by using a GC and not wanting to collect all the time, you need to add more memory overhead, which is not required with Rust. Comparison: I have one app written with Scala and consuming 350 megabytes per Mesos task. The similar-sized Rust app is using 8-10 megabytes of RAM. When you have 1000 instances running during a peak, this escalates fast.
  - reality_hacker 10y ago
    So, for jvm you create less tasks but with more ram, e.g. 100 tasks 4G RAM each, and problem is solved?
    - pimeys 10y ago
      Having more of the smaller tasks spreads out the load more evenly and when rust apps starts in less than a second, it's easy to fine tune and reason about having more tasks running. With JVM, we just waste memory because of the overhead from GC, and I don't feel comfortable with the idea.
  - lmm 10y ago
    A bit odd to be running a VM on a VM. Did you consider using a Java "application server" or similar that can keep a VM running and run tasks within that?
webscalist 10y ago
link to source code? want to see what 2-weeks old rust programmer writes and how feasible it is to train our team
- pimeys 10y ago
  Nothing open source yet. I might take the metrics system out and publish it on Github when I've used it in another service and it is a bit more generalized.
Varriount 10y ago
Have you looked at languages like Nim [1]? Garbage collectors don't have to be of the stop-the-world variety. [1] http://nim-lang.org/ http://nim-lang.org/ http://nim-lang.org/docs/gc.html http://nim-lang.org/docs/gc.html
- TD-Linux 10y ago
  That does look like a stop-the-world garbage collector, just with a tunable maximum runtime.
cel1ne 10y ago
Why would you need to start up an additional JVM for every task. Can't one JVM run multiple tasks?
- gdw2 10y ago
  The mention of Mesos may imply that each instance would be a separate container.

by pimeys 10y ago

karterk 10y ago

On the other hand, as your system gets more complex, you will start missing tried and test libraries from some of the more mature languages you have mentioned. Maybe the extra time you have to spend writing that code will compensate for the extra memory you would have spent because of the JVM. Not taking anything away from Rust - I'm just giving an alternative perspective.

pimeys 10y ago
That is a risk we've analyzed and are willing to take. Scala can stay on the data side of our platform, but the real time part will be either Rust or C++14 (and Python where we don't need raw speed and can handle everything with one thread). For the libraries the situation is not that bad. Something I was missing from Scala side was Kamon. It is a really awesome library I've been using (and contributing) for some time. For Rust I needed to do my own solution with threads and a bit of macro love. It works, it is using a constant amount of ram and when we get our next Rust project running, I might open source that part for everybody to use...
- annnnd 10y ago
  As far as I can tell, Kamon is an application monitoring system - what do you monitor? I am curious as I have never felt the need to monitor apps this way (but I have never worked on HFT apps obviously).
  - pimeys 10y ago
    Counters, times and such. These are to be sent to a service like InfluxDB and monitored with a tool such as Grafana.
  - pimeys 10y ago
    Also, one thing I would love to see is the syntactic sugar for binding, like Haskell's `do` and Scala's `for`. Would be super useful for lots of things, e.g. validating requests in a web server.
loup-vaillant 10y ago
I bet a fair number of those tried and tested libraries expose a C interface. As far as I know, Rust have an FFI. Those libraries are only a set of bindings away. So I think we can do away with Java libraries, as well as the occasional header-only C++ template madness¹. 1: I mean, it's madness if you want to talk to it from the outside. If you stick with C++, this is "merely" flamewar material (take the Boost devs and the C++ committee on one side, and game devs like Mike Acton, Jonathan Blow, or Casey Muratori on the other side).
- lmm 10y ago
  > I bet a fair number of those tried and tested libraries expose a C interface. Some of them do, or have C equivalents, but many of them don't. Once you get into "there's only one library that does it and it's one guy's graduate thesis" territory, often the only option is on the JVM.
  - loup-vaillant 10y ago
    > "there's only one library that does it and it's one guy's graduate thesis" Those libraries aren't tried and tested. They can be reimplemented. And in some contexts, they shouldn't be trusted as a black box.
  - jleahy 10y ago
    The last time I was in that situation, the only option was lisp. I'm still supporting a lisp environment for that one piece of code.

gizzlon 10y ago

> Go I just don't like as a language (personal taste) and I think the GC adds a bit too much of pausing for this app Not saying you have to like Go, we all have different preferences. But FYI, the gc has gotten much better in the latest releases. I don't think you would be bothered by gc pauses. (my understanding is that it' both faster and more "spread out")

pimeys 10y ago
We're doing real time stuff where these pauses matter. Just that we don't really need GC for anything and adding it to certain services cause more harm than it causes good.
- gizzlon 10y ago
  Curious, how long can a pause be before it's a problem? (I do not write these kind of services)
  - jerf 10y ago
    "It depends". Some services are insensitive to pauses of several minutes (email processing; you may not want it there all the time but spikes to that level are often acceptable). Some services are sensitive to pauses in the single-digit milliseconds (for instance trying to maintain a high frame rate).
    - dllthomas 10y ago
      Two years ago I was working on a system that cared about single digit microseconds, luckily with low throughput.
      - jleahy 10y ago
        Lots of people have to worry about single digit microseconds with high throughput.

lmm 10y ago

Is the JVM overhead really significant these days? You're talking about wasting what, maybe 32MB for each instance? (which can have multiple threads, you don't need one per core). Is your RAM really so limited that you'd run out of RAM before maxing out all your CPUs or I/O in a Scala version? (I am kind of interested in Rust, but not going to touch it until it gets proper HKT support - you have to duplicate so much code otherwise)

pimeys 10y ago
It is a lot, when we need to scale by adding more Mesos tasks. Every task launches a new JVM instance, which adds to the total memory usage. Also by using a GC and not wanting to collect all the time, you need to add more memory overhead, which is not required with Rust. Comparison: I have one app written with Scala and consuming 350 megabytes per Mesos task. The similar-sized Rust app is using 8-10 megabytes of RAM. When you have 1000 instances running during a peak, this escalates fast.
- reality_hacker 10y ago
  So, for jvm you create less tasks but with more ram, e.g. 100 tasks 4G RAM each, and problem is solved?
  - pimeys 10y ago
    Having more of the smaller tasks spreads out the load more evenly and when rust apps starts in less than a second, it's easy to fine tune and reason about having more tasks running. With JVM, we just waste memory because of the overhead from GC, and I don't feel comfortable with the idea.
- lmm 10y ago
  A bit odd to be running a VM on a VM. Did you consider using a Java "application server" or similar that can keep a VM running and run tasks within that?

webscalist 10y ago

link to source code? want to see what 2-weeks old rust programmer writes and how feasible it is to train our team

pimeys 10y ago
Nothing open source yet. I might take the metrics system out and publish it on Github when I've used it in another service and it is a bit more generalized.

Varriount 10y ago

Have you looked at languages like Nim [1]? Garbage collectors don't have to be of the stop-the-world variety. [1] http://nim-lang.org/ http://nim-lang.org/ http://nim-lang.org/docs/gc.html http://nim-lang.org/docs/gc.html

TD-Linux 10y ago
That does look like a stop-the-world garbage collector, just with a tunable maximum runtime.

cel1ne 10y ago

Why would you need to start up an additional JVM for every task. Can't one JVM run multiple tasks?

gdw2 10y ago
The mention of Mesos may imply that each instance would be a separate container.