5 ms·

From the article: "Standing here it looks like Docker was invented to manage dependencies for Python, Javascript and Java. It looks strange from a platform tha

From the article: "Standing here it looks like Docker was invented to manage dependencies for Python, Javascript and Java. It looks strange from a platform that deploys as one single binary." Let me say the quiet part out loud: Docker is covering up the fact that we don't write deployable software any more. Go isn't perfect either. The author isn't dealing with assets (images anyone?). I think there is plenty of room for innovation here, and were over due for some change.

ElevenLathe 4y ago
Anymore? When were we writing "deployable" software in the past?
- marcosdumay 4y ago
  You used to buy a disk, put it on your computer and run the thing there.
  - ElevenLathe 4y ago
    Isn't that just "running" the software? To me, "deployment" implies some repeated process that actually wasn't especially valuable to automate or be very careful about when software was released once a year (and hence the reason it wasn't). Also, disks are a horrible way to deploy software. They have all the same problems of just distributing a random tarball: What operating system is it for? What version? Where do I copy the files? How do I get the OS to automatically start the service on startup? What version of make does it use? How about which libc and cc? You can say this stuff in the README (or printed docs) but isn't something more "deployable" when it's all machine-readable and can be reasoned about automatically? This is what package managers were invented for.
  - xyzzy4747 4y ago
    My father told me a story about when he was in college, he had to find a book in the library catalogue with the program he wanted, order it and check it out, and then type it up and test/debug it. After that all his colleagues and professors wanted to borrow it as well. This was in the 70s.
- zer00eyz 4y ago
  I'll bite! We used to ship software, on discs, and we didn't have the Internet to update it. There is plenty of software out there that works via your linux distress package manager. Note that this isn't everything you can get from your package manager. Plenty of things that are available are broken or miersable to get working unless you get the container/vm version.
  - tomcam 4y ago
    I remember those days well. It really meant you had to be careful with bugs and documentation. It is not clear to me that the we are winning with daily or multiple times daily release schedules at this point.
    - jrockway 4y ago
      Were people careful with bugs and documentation? I remember the Internet blowing up one day because every Windows install was sending every IP on the Internet a virus, and there was nothing anyone could do about it. (And yes, Unix also had similar worms, though they largely predate me!) Word used to crash and corrupt your entire novel. There was no online banking. I'm not sure the rose-colored glasses are a realistic take on changing software quality. Today, the tools are available to move quickly and maintain quality. You probably do what was months of manual testing every time you save a file, and certainly every time you commit a set of changes. There are fuzz testers to find the craziest bugs that no human could even imagine. There are robots that read your PR and point out common errors. HN really likes to pan on software quality, but "I don't like this feature" is not a bug per se, just a company you don't like. There are a lot of those, but there are more lines of code than ever, and a lot more stuff works than 30 years ago. I think we, as a field, are getting better.
      - tomcam 4y ago
        I ran a software company that predated the web. Yes, we were incredibly careful. When you ran the risk of having to send out updates on disks QC was a big thing.
- Jtsummers 4y ago
  A better question is what is deployable software? How does it contrast from non-deployable so we can understand what we're even talking about. Software gets "deployed" all the time so in what way is it currently non-deployable versus some rose tinted view of yesteryear's software?
galdor 4y ago
I'm unsatisfied with the current situation too, but it's a hard problem. You can either go full container (which means no BSD, and having to deal with Docker or Kubernetes and all the associated woes), or fallback to native packages which are a huge PITA to build, deploy and use. I think that Nix and Guix have part of the solution: have a way to build fully independent packages that can be easily installed. But I'm not comfortable with the complexity of Nix, and Guix does not run on FreeBSD. And ultimately you still have to handle distribution and configuration of the base system you deploy on. Innovation is possible, but there are a lot of expectations for any system dealing with building and deploying software. I feel that there are fundamental limitations inherited from the way UNIX OS work, and I wish we had lower level operating systems focused on executing services on multiple machines in a way similar to how mainframes work. One can dream.
- indymike 4y ago
  I'm really growing tired of significant effort development effort going to dealing with deployment on the part of our stack that isn't written in Go. The Go side, deployment is replace binary, restart app, done. The Python and Javascript code we maintain takes significant effort to deploy, and builds can be brittle due to dependency issues. > I feel that there are fundamental limitations inherited from the way UNIX OS work There are, but the way go does deployments plays to Unix's strengths.
- xyzzy4747 4y ago
  Just curious, what’s the benefit of using FreeBSD versus Debian/Ubuntu?
  - lmm 4y ago
    Not the person you replied to, but it tends to be more stable (commands and interfaces change less) and have better backwards compatibility; also ZFS is better-integrated than on Linux and better than any of the other options on Linux. (Jails used to be another advantage, but these days linux containers can more or less do most of the same things)
- lmm 4y ago
joncfoo 4y ago
Assets of any kind can be embedded in the executable and accessed via the embed.FS interface. This makes it trivial to bundle up all dependencies if desired.
- cytzol 4y ago
  Embedding your assets like this isn't always an improvement. For example, I work on a site with a Go server and static content pages, and I like that I can update one of the pages and see the change instantly without having to re-compile the entire server binary just to get the new files included.
  - mftb 4y ago
    Easy enough to have the app check the regular file system first, then fallback to the embedded fs. You could have the best of both worlds.
- candiodari 4y ago
  Of course Visual Basic 2.0 and Delphi 1.0 both had embeddable filesystems. Even updateable embedded filesystems (which worked because the exe would really be a zip files. Zip files are indexed from the end of the file. So you can prepend the actual executable code and that would work. Zip files are updateable ...) I believe after a while you also had sqlite-inside-the-exe things.
bojanz 4y ago
Go isn't perfect either. The author isn't dealing with assets (images anyone?). Go supports embedding assets since 1.16, the author mentions embedfs in the post.
- divan 4y ago
  And people are using go-bindata since Go<1.0, so Go supports embedding assets since forever.
acatton 4y ago
> Go isn't perfect either. The author isn't dealing with assets (images anyone?). From the article: "The Go web application had all files like configurations (no credentials), static css and html templates embedded with embedfs (and proxied through a CDN)." See https://pkg.go.dev/embed https://pkg.go.dev/embed
- zer00eyz 4y ago
  OP here, How are you doing caching without a mod time? https://github.com/golang/go/issues/44854 https://github.com/golang/go/issues/44854 Are you re-naming or hashing for cache clearing?
  - acatton 4y ago
    I'm not the author of the post, so I can't tell you what the author does. What I do in my projects is that I tell varnish-cache to cache assets in "/static/..." forever. And I have a "curl -X PURGE <varnish_endpoint>" as part of the "ExecStartPre=" of my go binary. • https://varnish-cache.org/docs/trunk/users-guide/purging.html#http-purging https://varnish-cache.org/docs/trunk/users-guide/purging.htm... • https://www.freedesktop.org/software/systemd/man/systemd.service.html#ExecStartPre= https://www.freedesktop.org/software/systemd/man/systemd.ser...
    - ed25519FUUU 4y ago
      Clever solution! Why not make it part of the startup of your binary?
      - acatton 4y ago
        Sorry to reply a day later. GP here, it's a simple separation of concern. IMHO the binary should not be aware that it is cached and/or how it's deployed. I separate "business logic" from "devops", and I consider purging the cache to be "devops". This is why I let systemd do it. Of course, at the end it's a question of preference. People might disagree.
deleted 4y ago
[deleted]
pjmlp 4y ago
Java never needed Docker, in fact the Docker / Kubernetes is now re-inventing Java Application Servers with WASM containers.
- VBprogrammer 4y ago
  XML ruined Java.
  - pjmlp 4y ago
    Says the guy with a nickname from a language that has XML support on its type system.
    - VBprogrammer 4y ago
      Not back in the good days of VB6 when I adopted the nickname.
Rapzid 4y ago
> we don't write deployable software any more What did it use to look like exactly, this "deployable" software? Going back to the birth of web 2.0 we had Perl, PHP, Java(?), .Net Framework a few years later. These all required tons of pre-configured infrastructure on the servers to run.. > It looks strange from a platform that deploys as one single binary It's just a tool with many uses. I CAN deploy my Asp.Net app as a self-contained(even single) file.. But the size of updates is smaller between images if I copy the app into an image that already has the .Net and Asp.Net library code in the base layers.
- ed25519FUUU 4y ago
  You can do that without ever touching the system libraries?
golergka 4y ago
> Docker is covering up the fact that we don't write deployable software any more. We do. It's just software now is a container image, and Docker is a development tool like Make.

ElevenLathe 4y ago

Anymore? When were we writing "deployable" software in the past?

marcosdumay 4y ago
You used to buy a disk, put it on your computer and run the thing there.
- ElevenLathe 4y ago
  Isn't that just "running" the software? To me, "deployment" implies some repeated process that actually wasn't especially valuable to automate or be very careful about when software was released once a year (and hence the reason it wasn't). Also, disks are a horrible way to deploy software. They have all the same problems of just distributing a random tarball: What operating system is it for? What version? Where do I copy the files? How do I get the OS to automatically start the service on startup? What version of make does it use? How about which libc and cc? You can say this stuff in the README (or printed docs) but isn't something more "deployable" when it's all machine-readable and can be reasoned about automatically? This is what package managers were invented for.
- xyzzy4747 4y ago
  My father told me a story about when he was in college, he had to find a book in the library catalogue with the program he wanted, order it and check it out, and then type it up and test/debug it. After that all his colleagues and professors wanted to borrow it as well. This was in the 70s.
zer00eyz 4y ago
I'll bite! We used to ship software, on discs, and we didn't have the Internet to update it. There is plenty of software out there that works via your linux distress package manager. Note that this isn't everything you can get from your package manager. Plenty of things that are available are broken or miersable to get working unless you get the container/vm version.
- tomcam 4y ago
  I remember those days well. It really meant you had to be careful with bugs and documentation. It is not clear to me that the we are winning with daily or multiple times daily release schedules at this point.
  - jrockway 4y ago
    Were people careful with bugs and documentation? I remember the Internet blowing up one day because every Windows install was sending every IP on the Internet a virus, and there was nothing anyone could do about it. (And yes, Unix also had similar worms, though they largely predate me!) Word used to crash and corrupt your entire novel. There was no online banking. I'm not sure the rose-colored glasses are a realistic take on changing software quality. Today, the tools are available to move quickly and maintain quality. You probably do what was months of manual testing every time you save a file, and certainly every time you commit a set of changes. There are fuzz testers to find the craziest bugs that no human could even imagine. There are robots that read your PR and point out common errors. HN really likes to pan on software quality, but "I don't like this feature" is not a bug per se, just a company you don't like. There are a lot of those, but there are more lines of code than ever, and a lot more stuff works than 30 years ago. I think we, as a field, are getting better.
    - tomcam 4y ago
      I ran a software company that predated the web. Yes, we were incredibly careful. When you ran the risk of having to send out updates on disks QC was a big thing.
Jtsummers 4y ago
A better question is what is deployable software? How does it contrast from non-deployable so we can understand what we're even talking about. Software gets "deployed" all the time so in what way is it currently non-deployable versus some rose tinted view of yesteryear's software?

galdor 4y ago

I'm unsatisfied with the current situation too, but it's a hard problem. You can either go full container (which means no BSD, and having to deal with Docker or Kubernetes and all the associated woes), or fallback to native packages which are a huge PITA to build, deploy and use. I think that Nix and Guix have part of the solution: have a way to build fully independent packages that can be easily installed. But I'm not comfortable with the complexity of Nix, and Guix does not run on FreeBSD. And ultimately you still have to handle distribution and configuration of the base system you deploy on. Innovation is possible, but there are a lot of expectations for any system dealing with building and deploying software. I feel that there are fundamental limitations inherited from the way UNIX OS work, and I wish we had lower level operating systems focused on executing services on multiple machines in a way similar to how mainframes work. One can dream.

indymike 4y ago
I'm really growing tired of significant effort development effort going to dealing with deployment on the part of our stack that isn't written in Go. The Go side, deployment is replace binary, restart app, done. The Python and Javascript code we maintain takes significant effort to deploy, and builds can be brittle due to dependency issues. > I feel that there are fundamental limitations inherited from the way UNIX OS work There are, but the way go does deployments plays to Unix's strengths.
xyzzy4747 4y ago
Just curious, what’s the benefit of using FreeBSD versus Debian/Ubuntu?
- lmm 4y ago
  Not the person you replied to, but it tends to be more stable (commands and interfaces change less) and have better backwards compatibility; also ZFS is better-integrated than on Linux and better than any of the other options on Linux. (Jails used to be another advantage, but these days linux containers can more or less do most of the same things)
lmm 4y ago

joncfoo 4y ago

Assets of any kind can be embedded in the executable and accessed via the embed.FS interface. This makes it trivial to bundle up all dependencies if desired.

cytzol 4y ago
Embedding your assets like this isn't always an improvement. For example, I work on a site with a Go server and static content pages, and I like that I can update one of the pages and see the change instantly without having to re-compile the entire server binary just to get the new files included.
- mftb 4y ago
  Easy enough to have the app check the regular file system first, then fallback to the embedded fs. You could have the best of both worlds.
candiodari 4y ago
Of course Visual Basic 2.0 and Delphi 1.0 both had embeddable filesystems. Even updateable embedded filesystems (which worked because the exe would really be a zip files. Zip files are indexed from the end of the file. So you can prepend the actual executable code and that would work. Zip files are updateable ...) I believe after a while you also had sqlite-inside-the-exe things.

bojanz 4y ago

Go isn't perfect either. The author isn't dealing with assets (images anyone?). Go supports embedding assets since 1.16, the author mentions embedfs in the post.

divan 4y ago
And people are using go-bindata since Go<1.0, so Go supports embedding assets since forever.

acatton 4y ago

> Go isn't perfect either. The author isn't dealing with assets (images anyone?). From the article: "The Go web application had all files like configurations (no credentials), static css and html templates embedded with embedfs (and proxied through a CDN)." See https://pkg.go.dev/embed https://pkg.go.dev/embed

zer00eyz 4y ago
OP here, How are you doing caching without a mod time? https://github.com/golang/go/issues/44854 https://github.com/golang/go/issues/44854 Are you re-naming or hashing for cache clearing?
- acatton 4y ago
  I'm not the author of the post, so I can't tell you what the author does. What I do in my projects is that I tell varnish-cache to cache assets in "/static/..." forever. And I have a "curl -X PURGE <varnish_endpoint>" as part of the "ExecStartPre=" of my go binary. • https://varnish-cache.org/docs/trunk/users-guide/purging.html#http-purging https://varnish-cache.org/docs/trunk/users-guide/purging.htm... • https://www.freedesktop.org/software/systemd/man/systemd.service.html#ExecStartPre= https://www.freedesktop.org/software/systemd/man/systemd.ser...
  - ed25519FUUU 4y ago
    Clever solution! Why not make it part of the startup of your binary?
    - acatton 4y ago
      Sorry to reply a day later. GP here, it's a simple separation of concern. IMHO the binary should not be aware that it is cached and/or how it's deployed. I separate "business logic" from "devops", and I consider purging the cache to be "devops". This is why I let systemd do it. Of course, at the end it's a question of preference. People might disagree.

deleted 4y ago

[deleted]

pjmlp 4y ago

Java never needed Docker, in fact the Docker / Kubernetes is now re-inventing Java Application Servers with WASM containers.

VBprogrammer 4y ago
XML ruined Java.
- pjmlp 4y ago
  Says the guy with a nickname from a language that has XML support on its type system.
  - VBprogrammer 4y ago
    Not back in the good days of VB6 when I adopted the nickname.

Rapzid 4y ago

> we don't write deployable software any more What did it use to look like exactly, this "deployable" software? Going back to the birth of web 2.0 we had Perl, PHP, Java(?), .Net Framework a few years later. These all required tons of pre-configured infrastructure on the servers to run.. > It looks strange from a platform that deploys as one single binary It's just a tool with many uses. I CAN deploy my Asp.Net app as a self-contained(even single) file.. But the size of updates is smaller between images if I copy the app into an image that already has the .Net and Asp.Net library code in the base layers.

ed25519FUUU 4y ago
You can do that without ever touching the system libraries?

golergka 4y ago

> Docker is covering up the fact that we don't write deployable software any more. We do. It's just software now is a container image, and Docker is a development tool like Make.