9 ms·

When I was stuck doing a web application in Java 15 years ago, I hated everything about it except for the deployment story, which boiled down to a single .war f

by masto 4y ago

When I was stuck doing a web application in Java 15 years ago, I hated everything about it except for the deployment story, which boiled down to a single .war file being pushed to the server. When we upgraded to Perl, I liked that system so we designed deployment around "PAR" files in a similar way, bundling all of the dependencies together with the application in the CI build process, and I wrote a tiny bit of infrastructure that essentially moved a symlink to make the new version live. Google uses MPM and hermetic packaging of configuration with binaries: https://sre.google/sre-book/release-engineering/#packaging-8ksQTeum https://sre.google/sre-book/release-engineering/#packaging-8... The way I see it, Docker is basically this same thing, generalized to be independent of the language/application/platform. As a practical matter, it still fundamentally has the "one file" nature. I don't see what's special or better about compiling everything into a single binary, apart from fetishizing the executable format. In any system at scale, you still have to solve for the more important problems of managing the infrastructure. "I can deploy by scping the file from my workstation to the server" is kind of a late 90s throwback, but golang is a 70s throwback, so I guess it fits?

marcosdumay 4y ago
> I don't see what's special or better about compiling everything into a single binary, apart from fetishizing the executable format. When you distribute your software to other people, it cuts the step of installing the correct interpreter... at the cost of requiring the correct computer architecture. It is very likely a gain.
- somehnguy 4y ago
  Exactly - I primarily write Java and fat-jars are great when developing apps for environments I control. But if I want to send an app to a friend it's a few additional steps to make sure they have the correct version of Java, paths are setup correct, etc. This isn't always trivial if they already have a different version of Java and want things to play nice side by side. Just bundle everything into a native executable, so many little annoyances just disappear. From what I understand Java does have facilities to bundle the runtime now but I haven't had the opportunity to really play with it yet.
  - vips7L 4y ago
    > From what I understand Java does have facilities to bundle jlink, jpackage, and if you need something complex you can use conveyor: https://hydraulic.software/index.html https://hydraulic.software/index.html
    - mike_hearn 4y ago
      Or even if you need something simple :-) Sending little apps to friends is easy now just like it once was, the difference is that the friends don't need to know what runtime you use. Also not only for Java: Electron, Flutter and anything else works too. We have an internal version of Conveyor that can be used to push servers as well. It bundles the jvm, makes debs with auto-scanned dependencies, systemd integration is automatically set up, it's easy to run the server with the DynamicUser feature for sandboxing and it uploads/installs the packages for you via ssh. We use it for our own servers. There seems to be more interest these days in running servers without big cloud overheads, so maybe we should launch it? It's hard to know how much demand there is for this sort of thing, though it gets rid of the hassle of manually configuring systemd and copying files around.
  - ihateolives 4y ago
    > But if I want to send an app to a friend it's a few additional steps to make sure they have the correct version of Java, paths are setup correct, etc My friends would do full stop and reverse at "install Java" step. It will just not fly with 99% of people. It's not 1999 anymore.
  - lenkite 4y ago
    You can use Graal Native Image https://www.graalvm.org/22.0/reference-manual/native-image/ https://www.graalvm.org/22.0/reference-manual/native-image/ to produce a single native executable. Example of a Java Micro-service framework that has first class support for this is Quarkus (https://quarkus.io/ https://quarkus.io/). See https://quarkus.io/guides/building-native-image https://quarkus.io/guides/building-native-image For a plain Java app: "Use Maven to Build a Native Executable from a Java Application" https://www.graalvm.org/22.2/reference-manual/native-image/guides/use-native-image-maven-plugin/ https://www.graalvm.org/22.2/reference-manual/native-image/g...
- busterarm 4y ago
  Also if you had some catastrophe where you're no longer able to build overnight or if you have to replace 100% of your infrastructure, you're still able to operate because you have a single compiled binary to ship. It eliminates whole classes of business risk. The more hosts in your fleet the more risk eliminated as well.
- JCWasmx86 4y ago
  You could take it a step further and make the user download a small stub Actually Portable Executable (https://justine.lol/ape.html https://justine.lol/ape.html) which downloads the real binaries.
- eddsh1994 4y ago
  > it cuts the step of installing the correct interpreter... at the cost of requiring the correct computer architecture. Obviously this depends on the product but I'd give anything to worry about interpreters over the correct computer architecture in the M1/M2 Intel Embedded world.
- coldtea 4y ago
  >When you distribute your software to other people, it cuts the step of installing the correct interpreter... at the cost of requiring the correct computer architecture Not even, as it's trivial to cross compile on Golang. Then you just offer 3-4 arch binaries, and they download the one that matches their platform.
- davemp 4y ago
  > It is very likely a gain. Distributing python has to be some of the worst experiences I’ve had in the field.
- lmm 4y ago
  Not convinced, not if you built for e.g. Java 8. I think there's a decent chance there are more people running something other than x86_64 nowadays (and that number's only going up) than people who don't have a JVM installed.
pjc50 4y ago
> I don't see what's special or better about compiling everything into a single binary, apart from fetishizing the executable format. Indeed. If you think of the docker image itself as an executable format like PE or ELF, this becomes clearer. Rather than targeting the OS API, which has completely the wrong set of security abstractions because it's built around "users", it defines a new API layer. > "I can deploy by scping the file from my workstation to the server" I kind of miss cgi-bin. If we're ever to get back to a place where random "power users" can knock up a quick server to meet some computing need they have, easy deployment has to be a big part of that. Can we make it as easy to deploy as to post on Instagram?
- mrkeen 4y ago
  What does cgi-bin get you? What does it run on? If I already have a single-file web server, do I need to introduce extra cgi-bin technology also?
  - pjc50 4y ago
    Apache supported it, back in the day before nginix existed. It gives you a single-file web page. You can then have multiple pages on the same server run by different users under different userids. It also operates on a model of one execution run per request. So CGIs that aren't currently being served consume no resources.
    - mike_hearn 4y ago
      It still does support it. In fact I use it on our company website to handle the contact form submission. It invokes a Kotlin script which reads the form, handles the recaptcha call and sends an email. Old school but it works and doesn't require any resources except when running.
  - jacobr1 4y ago
    For anybody comfortable with the compiling the single-file binary ... it doesn't gain you anything. For a class of "power-end-users" it provides a mechanism to build sandboxed apps on a multi-tenant system. I think the spiritual successors split though between PAAS/heroku-like systems and "low-code" platforms.
justinsaccount 4y ago
Have you ever read https://medium.com/@gtrevorjay/you-could-have-invented-container-runtimes-an-explanatory-fantasy-764c5b389bd3 https://medium.com/@gtrevorjay/you-could-have-invented-conta... ? You were halfway there :-)
adql 4y ago
Well, system-wise Go app is just a binary that only needs network access, could be run directly from systemd and just have all permissions set there. Docker is a bunch of file mounts and app running in separate namespaces. So extra daemon, extra layers of complexity. Of course if you're already deploying other docker apps it doesn't really matter, as you'd want to have that one binary in docker container anyway just to manage everything from same place.
- BrandoElFollito 4y ago
  There is also the deployment part that is easier (au least for an amateur dev such as myself). I have a CI/CD template, Erin all my web stuff via a dockerized caddy reverse proxy, do not need to touch the configuration of the host (to create .service &co. files) I find deploying to docker just simpler.
veidelis 4y ago
Excellent roast!

by masto 4y ago

marcosdumay 4y ago

> I don't see what's special or better about compiling everything into a single binary, apart from fetishizing the executable format. When you distribute your software to other people, it cuts the step of installing the correct interpreter... at the cost of requiring the correct computer architecture. It is very likely a gain.

somehnguy 4y ago
Exactly - I primarily write Java and fat-jars are great when developing apps for environments I control. But if I want to send an app to a friend it's a few additional steps to make sure they have the correct version of Java, paths are setup correct, etc. This isn't always trivial if they already have a different version of Java and want things to play nice side by side. Just bundle everything into a native executable, so many little annoyances just disappear. From what I understand Java does have facilities to bundle the runtime now but I haven't had the opportunity to really play with it yet.
- vips7L 4y ago
  > From what I understand Java does have facilities to bundle jlink, jpackage, and if you need something complex you can use conveyor: https://hydraulic.software/index.html https://hydraulic.software/index.html
  - mike_hearn 4y ago
    Or even if you need something simple :-) Sending little apps to friends is easy now just like it once was, the difference is that the friends don't need to know what runtime you use. Also not only for Java: Electron, Flutter and anything else works too. We have an internal version of Conveyor that can be used to push servers as well. It bundles the jvm, makes debs with auto-scanned dependencies, systemd integration is automatically set up, it's easy to run the server with the DynamicUser feature for sandboxing and it uploads/installs the packages for you via ssh. We use it for our own servers. There seems to be more interest these days in running servers without big cloud overheads, so maybe we should launch it? It's hard to know how much demand there is for this sort of thing, though it gets rid of the hassle of manually configuring systemd and copying files around.
- ihateolives 4y ago
  > But if I want to send an app to a friend it's a few additional steps to make sure they have the correct version of Java, paths are setup correct, etc My friends would do full stop and reverse at "install Java" step. It will just not fly with 99% of people. It's not 1999 anymore.
- lenkite 4y ago
  You can use Graal Native Image https://www.graalvm.org/22.0/reference-manual/native-image/ https://www.graalvm.org/22.0/reference-manual/native-image/ to produce a single native executable. Example of a Java Micro-service framework that has first class support for this is Quarkus (https://quarkus.io/ https://quarkus.io/). See https://quarkus.io/guides/building-native-image https://quarkus.io/guides/building-native-image For a plain Java app: "Use Maven to Build a Native Executable from a Java Application" https://www.graalvm.org/22.2/reference-manual/native-image/guides/use-native-image-maven-plugin/ https://www.graalvm.org/22.2/reference-manual/native-image/g...
busterarm 4y ago
Also if you had some catastrophe where you're no longer able to build overnight or if you have to replace 100% of your infrastructure, you're still able to operate because you have a single compiled binary to ship. It eliminates whole classes of business risk. The more hosts in your fleet the more risk eliminated as well.
JCWasmx86 4y ago
You could take it a step further and make the user download a small stub Actually Portable Executable (https://justine.lol/ape.html https://justine.lol/ape.html) which downloads the real binaries.
eddsh1994 4y ago
> it cuts the step of installing the correct interpreter... at the cost of requiring the correct computer architecture. Obviously this depends on the product but I'd give anything to worry about interpreters over the correct computer architecture in the M1/M2 Intel Embedded world.
coldtea 4y ago
>When you distribute your software to other people, it cuts the step of installing the correct interpreter... at the cost of requiring the correct computer architecture Not even, as it's trivial to cross compile on Golang. Then you just offer 3-4 arch binaries, and they download the one that matches their platform.
davemp 4y ago
> It is very likely a gain. Distributing python has to be some of the worst experiences I’ve had in the field.
lmm 4y ago
Not convinced, not if you built for e.g. Java 8. I think there's a decent chance there are more people running something other than x86_64 nowadays (and that number's only going up) than people who don't have a JVM installed.

pjc50 4y ago

> I don't see what's special or better about compiling everything into a single binary, apart from fetishizing the executable format. Indeed. If you think of the docker image itself as an executable format like PE or ELF, this becomes clearer. Rather than targeting the OS API, which has completely the wrong set of security abstractions because it's built around "users", it defines a new API layer. > "I can deploy by scping the file from my workstation to the server" I kind of miss cgi-bin. If we're ever to get back to a place where random "power users" can knock up a quick server to meet some computing need they have, easy deployment has to be a big part of that. Can we make it as easy to deploy as to post on Instagram?

mrkeen 4y ago
What does cgi-bin get you? What does it run on? If I already have a single-file web server, do I need to introduce extra cgi-bin technology also?
- pjc50 4y ago
  Apache supported it, back in the day before nginix existed. It gives you a single-file web page. You can then have multiple pages on the same server run by different users under different userids. It also operates on a model of one execution run per request. So CGIs that aren't currently being served consume no resources.
  - mike_hearn 4y ago
    It still does support it. In fact I use it on our company website to handle the contact form submission. It invokes a Kotlin script which reads the form, handles the recaptcha call and sends an email. Old school but it works and doesn't require any resources except when running.
- jacobr1 4y ago
  For anybody comfortable with the compiling the single-file binary ... it doesn't gain you anything. For a class of "power-end-users" it provides a mechanism to build sandboxed apps on a multi-tenant system. I think the spiritual successors split though between PAAS/heroku-like systems and "low-code" platforms.

justinsaccount 4y ago

Have you ever read https://medium.com/@gtrevorjay/you-could-have-invented-container-runtimes-an-explanatory-fantasy-764c5b389bd3 https://medium.com/@gtrevorjay/you-could-have-invented-conta... ? You were halfway there :-)

adql 4y ago

Well, system-wise Go app is just a binary that only needs network access, could be run directly from systemd and just have all permissions set there. Docker is a bunch of file mounts and app running in separate namespaces. So extra daemon, extra layers of complexity. Of course if you're already deploying other docker apps it doesn't really matter, as you'd want to have that one binary in docker container anyway just to manage everything from same place.

BrandoElFollito 4y ago
There is also the deployment part that is easier (au least for an amateur dev such as myself). I have a CI/CD template, Erin all my web stuff via a dockerized caddy reverse proxy, do not need to touch the configuration of the host (to create .service &co. files) I find deploying to docker just simpler.

veidelis 4y ago

Excellent roast!