3 ms·

It sure sounds like you are advocating Docker as a way to produce and distribute a http://martinfowler.com/bliki/SnowflakeServer.html http://martinfowler.com/

by sciurus 10y ago

It sure sounds like you are advocating Docker as a way to produce and distribute a http://martinfowler.com/bliki/SnowflakeServer.html http://martinfowler.com/bliki/SnowflakeServer.html We've been down this road before. Hacking something until it works, then imaging it and passing it on to your coworkers is a maintenance nightmare.

geocar 10y ago
I appreciate how difficult it is to understand someone when you begin the conversation by assuming you understand them. For example, did you read that article? You know Docker has none of those problems > The first problem with a snowflake server is that it's difficult to reproduce. Should your hardware start having problems, this means that it's difficult to fire up another server to support the same functions. docker tag/push/pull > If you need to run a cluster, you get difficulties keeping all of the instances of the cluster in sync. docker start tag > You can't easily mirror your production environment for testing. docker commit/export/import > When you get production faults, you can't investigate them by reproducing the transaction execution in a development environment. docker push/pull > The true fragility of snowflakes, however, comes when you need to change them. Snowflakes soon become hard to understand and modify. Upgrades of one bit software cause unpredictable knock-on effects. docker history/diff The issue that Martin is describing is one where people work on the server - update it in place, and do not have any change management process besides "do stuff" and any quality checking process besides "check stuff". He spent over 500 words to say that's probably not a good idea. No kidding. Now. What I'm actually advocating is interactive development: The exact same way we develop our APL workspaces and our Smalltalk images, can be used to design and build server infrastructure. Instead of a development cycle that goes: * Edit nixfile/dockerfile/vagrantfile * Wait (sometimes hours) * Test things * Repeat You get a very short: * Do stuff * Get happy * Tag the results Then when you want to run "in production", you publish your tags/image, review the history with your team, not dissimilar to how Smalltalk people will clean up their changes file, and then introduce the new version into the cluster. Once you're satisfied the world hasn't ended, you can decommission the old images. You can even make scripts that do these many of these steps automatically (divert x% of traffic to new instances, collect results, and so on). If you have a well-designed infrastructure, you can have junior developers use those scripts to make changes on their first day. Now. I get it, you have realised that if you start over often enough you get good at it. I get that you've seen people who treat systems development as a completely linear descent into unmaintainable bat barf that can only be recovered by a bad day reinstalling everything and "starting over", I also get that you think amortising that cost by taking those hours to rebuild your entire system when glibc changes is necessary to avoid that bad day. I also get you like static external configuration like Dockerfiles or Vagrantfiles, and view them as instrumental in the audit of that process. However while you understood that I disagree with that, you didn't understand why: If you had a decade or so of Smalltalk or Lisp experience instead of just ruby experience, you might have known exactly what I meant, but instead of asking, you just ignored the part you didn't understand, making a straw man argument that you did understand. After all, what's more likely, someone knows something important that you don't? Or someone knows something unimportant and just isn't as smart as you?
- pmahoney 10y ago
  I appreciate your presentation of an alternate viewpoint. The "build from source" camp provides solutions to a couple problems that at first glance seem to be difficult in a "principled mutation of binary image" model: - Moving to a different architecture, e.g. x86 to ARM or ppcle. In build from source, I essentially change some parameters and components in the lowest level and rebuild. - Other big changes to base components, e.g. compiling for Linux/libc vs. building atop a unikernel (obviously heavily dependent on what the upper layers are doing, but I can imagine wanting to run similar code atop multiple bases). - Sharing common components. This is not really a "from source" thing, but is an important part of Nix. If two images include components in different orders, in Nix the common components will be shared (at the arguably coarse granularity of a Nix package), while the linear structure of Docker layers means once two images diverge, everything further down the chain will be its own copy, even if the same bits.
  - geocar 10y ago
    "Sharing common components" is useful, and the fact that any combination of Nix packages is a valid combination is not something that is guaranteed by other package management systems[1]. However there's another, perhaps simpler solution: I can have a separate "wu-ftpd" container, and a separate "hylafax" container. This admittedly might not have been as easy when nix started, but it is very very easy now. [1]: https://cr.yp.to/slashpackage/studies.html https://cr.yp.to/slashpackage/studies.html

by sciurus 10y ago

geocar 10y ago

I appreciate how difficult it is to understand someone when you begin the conversation by assuming you understand them. For example, did you read that article? You know Docker has none of those problems > The first problem with a snowflake server is that it's difficult to reproduce. Should your hardware start having problems, this means that it's difficult to fire up another server to support the same functions. docker tag/push/pull > If you need to run a cluster, you get difficulties keeping all of the instances of the cluster in sync. docker start tag > You can't easily mirror your production environment for testing. docker commit/export/import > When you get production faults, you can't investigate them by reproducing the transaction execution in a development environment. docker push/pull > The true fragility of snowflakes, however, comes when you need to change them. Snowflakes soon become hard to understand and modify. Upgrades of one bit software cause unpredictable knock-on effects. docker history/diff The issue that Martin is describing is one where people work on the server - update it in place, and do not have any change management process besides "do stuff" and any quality checking process besides "check stuff". He spent over 500 words to say that's probably not a good idea. No kidding. Now. What I'm actually advocating is interactive development: The exact same way we develop our APL workspaces and our Smalltalk images, can be used to design and build server infrastructure. Instead of a development cycle that goes: * Edit nixfile/dockerfile/vagrantfile * Wait (sometimes hours) * Test things * Repeat You get a very short: * Do stuff * Get happy * Tag the results Then when you want to run "in production", you publish your tags/image, review the history with your team, not dissimilar to how Smalltalk people will clean up their changes file, and then introduce the new version into the cluster. Once you're satisfied the world hasn't ended, you can decommission the old images. You can even make scripts that do these many of these steps automatically (divert x% of traffic to new instances, collect results, and so on). If you have a well-designed infrastructure, you can have junior developers use those scripts to make changes on their first day. Now. I get it, you have realised that if you start over often enough you get good at it. I get that you've seen people who treat systems development as a completely linear descent into unmaintainable bat barf that can only be recovered by a bad day reinstalling everything and "starting over", I also get that you think amortising that cost by taking those hours to rebuild your entire system when glibc changes is necessary to avoid that bad day. I also get you like static external configuration like Dockerfiles or Vagrantfiles, and view them as instrumental in the audit of that process. However while you understood that I disagree with that, you didn't understand why: If you had a decade or so of Smalltalk or Lisp experience instead of just ruby experience, you might have known exactly what I meant, but instead of asking, you just ignored the part you didn't understand, making a straw man argument that you did understand. After all, what's more likely, someone knows something important that you don't? Or someone knows something unimportant and just isn't as smart as you?

pmahoney 10y ago
I appreciate your presentation of an alternate viewpoint. The "build from source" camp provides solutions to a couple problems that at first glance seem to be difficult in a "principled mutation of binary image" model: - Moving to a different architecture, e.g. x86 to ARM or ppcle. In build from source, I essentially change some parameters and components in the lowest level and rebuild. - Other big changes to base components, e.g. compiling for Linux/libc vs. building atop a unikernel (obviously heavily dependent on what the upper layers are doing, but I can imagine wanting to run similar code atop multiple bases). - Sharing common components. This is not really a "from source" thing, but is an important part of Nix. If two images include components in different orders, in Nix the common components will be shared (at the arguably coarse granularity of a Nix package), while the linear structure of Docker layers means once two images diverge, everything further down the chain will be its own copy, even if the same bits.
- geocar 10y ago
  "Sharing common components" is useful, and the fact that any combination of Nix packages is a valid combination is not something that is guaranteed by other package management systems[1]. However there's another, perhaps simpler solution: I can have a separate "wu-ftpd" container, and a separate "hylafax" container. This admittedly might not have been as easy when nix started, but it is very very easy now. [1]: https://cr.yp.to/slashpackage/studies.html https://cr.yp.to/slashpackage/studies.html