7 ms·

Never touch a running system ;) Scala on JVM is much more tested than the new shiny thing. Also don't expect improved performance... many people think that the

Never touch a running system ;) Scala on JVM is much more tested than the new shiny thing. Also don't expect improved performance... many people think that the JVM is bloated and makes programs slower (this is mostly not true). The downsides of the JVM are more memory consumption/footprint (when you have e.g. small servers or micro instances) and the cold startup time of the JVM itself (which is not relevant on a server in comparison to desktop Java apps). Would be interested to hear if any backend Scala projects like e.g. Play work on Scala Native.

bluejekyll 10y ago
Long time Java lover here. I agree with all your points, but in the context of Java at least (does Scala support this?) there is no simple static binary that can be built and released, which includes the JVM. I think 1.9 will have this option, but this is something I didn't realize I missed until I started work with Rust and Go. It makes deployment so much simpler.
- hocuspocus 10y ago
  In the age of containers it's really not that much harder to build and deploy a JVM app. Edit: thanks for the downvotes but you could at least tell me what's so crazy about my statement.
  - djsumdog 10y ago
    There are still some licensing issues. For instance, Atlassian has an official docker container to evaluate Confluence, but they don't support it in production since it uses OpenJDK and Confluence is still somewhat broken on OpenJDK. Rather than fix Confluence to work on OpenJDK (I don't want to imagine what type of reflection garbage they've got going on down there that breaks so bad on OpenJDK), their instructions tell you how to make your own Dockerfile using the official Oracle runtime. Actually, in that situation, if it won't run on OracleJDK it's probably not going to work via a native compiler either.
  - whatnotests 10y ago
    One app, sure. At work I'm running 17 different containers- many of which require their own JVM. (That's 3 different JRuby apps, zookeeper, kafka, and ElasticSearch.) Those JVMs get heavy when you're shipping container images compared to small Go or Rust binaries.
  - lmm 10y ago
    To my mind the JVM is where containers make the least sense. If you build an executable jar you can run with "java -jar ..." then that seems just as simple as "docker run ..." and gets you the single-file deployment, and you can control memory allocation via flags if you need to. You don't get virtual networking but IME that doesn't add value in the first place.
  - therealmarv 10y ago
    No downvote from me. But a explanation why e.g. compiled binaries are better. I had a hard time to get a normal non fancy Scala Play project running on a 512MB DigitalOcean instance. Mostly because it needs a lot more ram for building. I solved it with using a bigger swap partition. With single binary precompiled programs this problem is more a developer machine problem than a infrastructure problem. So I think the deployment step itself (and not looking at anything else) is easier with a small single binaries.
    - lmm 10y ago
      You shouldn't build it on the deployment server. You build a jar and upload/download that to/from the place you want it to run, just as you'd do with an executable. A jar is "not binary" but what practical difference does that make?
      - therealmarv 10y ago
        yes, you're right, thanks for pointing that out.
- pjmlp 10y ago
  Sure there is, all commercial JVMs support AOT compilation to native code. That most don't want to pay for them is another matter.
- lmm 10y ago
  Does it really need to be a binary? Build executable jars (use the maven shade plugin), run them with java -jar foo.jar, that's about as simple as it gets.
  - geodel 10y ago
    Maybe some apps do not have classpath. Here is what I see of running kafka instance on one of my server. And it does not looks like as simple as it gets. java -Xmx512M -Xms512M -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true -Xloggc:/opt/kafka_2.11-0.10.0.0/bin/../logs/zookeeper-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/opt/kafka_2.11-0.10.0.0/bin/../logs -Dlog4j.configuration=file:bin/../config/log4j.properties -cp :/opt/kafka_2.11-0.10.0.0/bin/../libs/aopalliance-repackaged-2.4.0-b34.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/argparse4j-0.5.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/connect-api-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/connect-file-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/connect-json-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/connect-runtime-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/guava-18.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/hk2-api-2.4.0-b34.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/hk2-locator-2.4.0-b34.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/hk2-utils-2.4.0-b34.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jackson-annotations-2.6.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jackson-core-2.6.3.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jackson-databind-2.6.3.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jackson-jaxrs-base-2.6.3.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jackson-jaxrs-json-provider-2.6.3.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jackson-module-jaxb-annotations-2.6.3.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/javassist-3.18.2-GA.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/javax.annotation-api-1.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/javax.inject-1.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/javax.inject-2.4.0-b34.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/javax.ws.rs-api-2.0.1.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-client-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-common-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-container-servlet-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-container-servlet-core-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-guava-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-media-jaxb-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-server-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-continuation-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-http-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-io-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-security-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-server-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-servlet-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-servlets-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-util-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jopt-simple-4.9.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka_2.11-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka_2.11-0.10.0.0-sources.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka_2.11-0.10.0.0-test-sources.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka-clients-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka-log4j-appender-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka-streams-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka-streams-examples-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka-tools-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/log4j-1.2.17.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/lz4-1.3.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/osgi-resource-locator-1.0.1.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/reflections-0.9.10.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/rocksdbjni-4.4.1.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/scala-library-2.11.8.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/scala-parser-combinators_2.11-1.0.4.jar:/opt/kafka_2.11-0.10.0.0/bin/../lib
algesten 10y ago
> the cold startup time of the JVM itself (which is not relevant on a server in comparison to desktop Java apps). I disagree somewhat with this. We found that when we started writing microservices in languages that are not java, the short startup time changed how we did some error handling. For errors where we say lose connection to the database, or rabbitmq, we much rather have the nodejs-process die and restart, than try to construct reconnect logic. The problem with reconnect-logic is that it is code that (may) be tested very rarely. This in turn means it's easy to get strange long term problems there like very slow memory leak due to some listener being added to a connection object once the connection is initiated. We did a 180 on reconnect-logic in our nodejs-processes and let the exceptions just bubble unhandled and take the entire vm down. With automatic restart script, the process will be back in seconds anyway, and with docker having built in back-off timers for auto restart, we don't necessarily overload the shared resources.
- mason55 10y ago
  I guess I don't see the difference here if your VM startup time is 3 seconds or 15-30 seconds. If that's the difference between the site remaining stable and the whole thing collapsing then it seems like you're setting yourself up for a big outage one day when the nodejs process isn't able to come back in three seconds for whatever reason.
  - algesten 10y ago
    I think it depends a bit on class of errors. Certainly not everything is suitable for this treatment. Lost connectivity to RabbitMQ or Elasticsearch would mean our site is dead anyhow (you can't do anything). So either of those errors should arguably result in some static 500 pardon-our-appearance page. But say someone messes up the network connection or we get a brief problem. Why wouldn't the nodejs process start quickly?
- wtetzner 10y ago
  > For errors where we say lose connection to the database, or rabbitmq, we much rather have the nodejs-process die and restart, than try to construct reconnect logic. This sounds like a very Erlang-ish way to handle the problem. Another advantage is that if the server/process is in some weird state that's causing problems, killing and restarting it lets you clear out the broken state, and get back into the state that it's most likely been tested under.
manojlds 10y ago
I'm interested in building CLIs with this.
acjohnson55 10y ago
At the moment, there isn't direct support for multithreading [1], so I'm guessing it would be very difficult to run any of the common web servers or computing frameworks natively. It may be possible for libraries that have pluggable concurrency, for example by creating an `ExecutionContext` that wraps OS threads, but that's waaay beyond my pay grade. [1] http://www.scala-native.org/en/latest/user/lang.html#lang http://www.scala-native.org/en/latest/user/lang.html#lang
- conjectures 10y ago
  Cheers - this is the #1 item on my excitement checklist.
dragonwriter 10y ago
> the cold startup time of the JVM itself (which is not relevant on a server in comparison to desktop Java apps). With microservices run as containers that are started on demand it matters; with other architectural choices it may matter less.

bluejekyll 10y ago

Long time Java lover here. I agree with all your points, but in the context of Java at least (does Scala support this?) there is no simple static binary that can be built and released, which includes the JVM. I think 1.9 will have this option, but this is something I didn't realize I missed until I started work with Rust and Go. It makes deployment so much simpler.

hocuspocus 10y ago
In the age of containers it's really not that much harder to build and deploy a JVM app. Edit: thanks for the downvotes but you could at least tell me what's so crazy about my statement.
- djsumdog 10y ago
  There are still some licensing issues. For instance, Atlassian has an official docker container to evaluate Confluence, but they don't support it in production since it uses OpenJDK and Confluence is still somewhat broken on OpenJDK. Rather than fix Confluence to work on OpenJDK (I don't want to imagine what type of reflection garbage they've got going on down there that breaks so bad on OpenJDK), their instructions tell you how to make your own Dockerfile using the official Oracle runtime. Actually, in that situation, if it won't run on OracleJDK it's probably not going to work via a native compiler either.
- whatnotests 10y ago
  One app, sure. At work I'm running 17 different containers- many of which require their own JVM. (That's 3 different JRuby apps, zookeeper, kafka, and ElasticSearch.) Those JVMs get heavy when you're shipping container images compared to small Go or Rust binaries.
- lmm 10y ago
  To my mind the JVM is where containers make the least sense. If you build an executable jar you can run with "java -jar ..." then that seems just as simple as "docker run ..." and gets you the single-file deployment, and you can control memory allocation via flags if you need to. You don't get virtual networking but IME that doesn't add value in the first place.
- therealmarv 10y ago
  No downvote from me. But a explanation why e.g. compiled binaries are better. I had a hard time to get a normal non fancy Scala Play project running on a 512MB DigitalOcean instance. Mostly because it needs a lot more ram for building. I solved it with using a bigger swap partition. With single binary precompiled programs this problem is more a developer machine problem than a infrastructure problem. So I think the deployment step itself (and not looking at anything else) is easier with a small single binaries.
  - lmm 10y ago
    You shouldn't build it on the deployment server. You build a jar and upload/download that to/from the place you want it to run, just as you'd do with an executable. A jar is "not binary" but what practical difference does that make?
    - therealmarv 10y ago
      yes, you're right, thanks for pointing that out.
pjmlp 10y ago
Sure there is, all commercial JVMs support AOT compilation to native code. That most don't want to pay for them is another matter.
lmm 10y ago
Does it really need to be a binary? Build executable jars (use the maven shade plugin), run them with java -jar foo.jar, that's about as simple as it gets.
- geodel 10y ago
  Maybe some apps do not have classpath. Here is what I see of running kafka instance on one of my server. And it does not looks like as simple as it gets. java -Xmx512M -Xms512M -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.awt.headless=true -Xloggc:/opt/kafka_2.11-0.10.0.0/bin/../logs/zookeeper-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/opt/kafka_2.11-0.10.0.0/bin/../logs -Dlog4j.configuration=file:bin/../config/log4j.properties -cp :/opt/kafka_2.11-0.10.0.0/bin/../libs/aopalliance-repackaged-2.4.0-b34.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/argparse4j-0.5.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/connect-api-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/connect-file-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/connect-json-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/connect-runtime-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/guava-18.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/hk2-api-2.4.0-b34.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/hk2-locator-2.4.0-b34.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/hk2-utils-2.4.0-b34.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jackson-annotations-2.6.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jackson-core-2.6.3.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jackson-databind-2.6.3.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jackson-jaxrs-base-2.6.3.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jackson-jaxrs-json-provider-2.6.3.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jackson-module-jaxb-annotations-2.6.3.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/javassist-3.18.2-GA.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/javax.annotation-api-1.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/javax.inject-1.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/javax.inject-2.4.0-b34.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/javax.ws.rs-api-2.0.1.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-client-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-common-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-container-servlet-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-container-servlet-core-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-guava-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-media-jaxb-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jersey-server-2.22.2.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-continuation-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-http-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-io-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-security-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-server-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-servlet-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-servlets-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jetty-util-9.2.15.v20160210.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/jopt-simple-4.9.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka_2.11-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka_2.11-0.10.0.0-sources.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka_2.11-0.10.0.0-test-sources.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka-clients-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka-log4j-appender-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka-streams-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka-streams-examples-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/kafka-tools-0.10.0.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/log4j-1.2.17.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/lz4-1.3.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/osgi-resource-locator-1.0.1.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/reflections-0.9.10.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/rocksdbjni-4.4.1.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/scala-library-2.11.8.jar:/opt/kafka_2.11-0.10.0.0/bin/../libs/scala-parser-combinators_2.11-1.0.4.jar:/opt/kafka_2.11-0.10.0.0/bin/../lib

algesten 10y ago

> the cold startup time of the JVM itself (which is not relevant on a server in comparison to desktop Java apps). I disagree somewhat with this. We found that when we started writing microservices in languages that are not java, the short startup time changed how we did some error handling. For errors where we say lose connection to the database, or rabbitmq, we much rather have the nodejs-process die and restart, than try to construct reconnect logic. The problem with reconnect-logic is that it is code that (may) be tested very rarely. This in turn means it's easy to get strange long term problems there like very slow memory leak due to some listener being added to a connection object once the connection is initiated. We did a 180 on reconnect-logic in our nodejs-processes and let the exceptions just bubble unhandled and take the entire vm down. With automatic restart script, the process will be back in seconds anyway, and with docker having built in back-off timers for auto restart, we don't necessarily overload the shared resources.

mason55 10y ago
I guess I don't see the difference here if your VM startup time is 3 seconds or 15-30 seconds. If that's the difference between the site remaining stable and the whole thing collapsing then it seems like you're setting yourself up for a big outage one day when the nodejs process isn't able to come back in three seconds for whatever reason.
- algesten 10y ago
  I think it depends a bit on class of errors. Certainly not everything is suitable for this treatment. Lost connectivity to RabbitMQ or Elasticsearch would mean our site is dead anyhow (you can't do anything). So either of those errors should arguably result in some static 500 pardon-our-appearance page. But say someone messes up the network connection or we get a brief problem. Why wouldn't the nodejs process start quickly?
wtetzner 10y ago
> For errors where we say lose connection to the database, or rabbitmq, we much rather have the nodejs-process die and restart, than try to construct reconnect logic. This sounds like a very Erlang-ish way to handle the problem. Another advantage is that if the server/process is in some weird state that's causing problems, killing and restarting it lets you clear out the broken state, and get back into the state that it's most likely been tested under.

manojlds 10y ago

I'm interested in building CLIs with this.

acjohnson55 10y ago

At the moment, there isn't direct support for multithreading [1], so I'm guessing it would be very difficult to run any of the common web servers or computing frameworks natively. It may be possible for libraries that have pluggable concurrency, for example by creating an `ExecutionContext` that wraps OS threads, but that's waaay beyond my pay grade. [1] http://www.scala-native.org/en/latest/user/lang.html#lang http://www.scala-native.org/en/latest/user/lang.html#lang

conjectures 10y ago
Cheers - this is the #1 item on my excitement checklist.

dragonwriter 10y ago

> the cold startup time of the JVM itself (which is not relevant on a server in comparison to desktop Java apps). With microservices run as containers that are started on demand it matters; with other architectural choices it may matter less.