5 ms·

We wrote an article about the same thing https://blog.wearewizards.io/why-docker-is-not-the-answer-to-reproducible-research-and-why-nix-may-be https://blog.wear

by Keats 10y ago

We wrote an article about the same thing https://blog.wearewizards.io/why-docker-is-not-the-answer-to-reproducible-research-and-why-nix-may-be https://blog.wearewizards.io/why-docker-is-not-the-answer-to... (Tom has a background in physics as well)

geocar 10y ago
You are assuming "that reproducibility is good". I believe this assumption needs to be challenged.
- coldtea 10y ago
  Any significant contribution you might have for software engineering in regards to that? Aside from merely believing that. I can't even tell what you mean. If I want to rebuild my machine same as before, or deploy build 100s of servers at different points in time with the same setup, reproducibility is necessary. There's not even one argument against that. If you have any, please oblige us. Should deployments be a surprise?
  - geocar 10y ago
    > If I want to rebuild my machine same as before, or deploy build 100s of servers at different points in time with the same setup, reproducibility is necessary. No it isn't. Reproducible is talking about the process or steps needed to make a machine. You do not need to repeat the steps to produce the first machine to produce the second machine. You can simply clone the first machine, and Docker makes this very easy to do. Keats begins his article with the statement offered, without justification: In this post I'm going to assume that reproducibility is good and necessary. Since I don't believe that, and in fact believe the opposite, I take issue with the entire article. Indeed, he goes on to construct a straw man comparing the exact opposite of what I propose -- a non-interactive "dockerfile" and not even a particularly good one, with a very well thought-out "purely functional" build system. > Any significant contribution you might have for software engineering in regards to that? I don't know of any big technology company that doesn't design their live infrastructure around ephemeral machines cloned from a template. They don't all use Docker, and they don't all use interactive development, however, but these two tools can make it easier for people who (for example) don't have Google's economy of scale. > Should deployments be a surprise? No they should not. Deployments take less time with my interactive method because you do not have to wait to "rebuild everything" which can take hours. Remember heartbleed? People using nixos had to rebuild every component that used OpenSSL, then test them, then find something else broke and repeat the process. Only once they had finished building their new process could they roll out this process to all machines, doing one more final rebuild. Even if done perfectly this has a minimum of two iterations. I could simply create a new clone, get it working, then tag it as the next release. New instances can then be cloned from the release, monitored, and the old instances destroyed. The entire process took under an hour for the entire fleet, including reading about heartbleed on HN. This is clearly almost as fast as running `apt-get dist-upgrade` on each of the live systems, but without any of the risk. The organisational complexity, however is admittedly much higher than "wash, wait, repeat", which is why most system administrators could not even begin to use live clones until virtualization and tooling had become much more commonplace. > I can't even tell what you mean. And yet you feel qualified to tell me I'm wrong.
    - coldtea 10y ago
      >Reproducible is talking about the process or steps needed to make a machine. You do not need to repeat the steps to produce the first machine to produce the second machine. You can simply clone the first machine, and Docker makes this very easy to do. Cloning is not the answer since it's all or nothing -- where reproducibility can be achieved piecemeal. Only offering cloning means that starting with the same initial Dockerfile and making some needed updates to it, one can't be sure that the NON updated parts will be producible. So you don't get flexible reuse. Only offering cloning also means that when you need to update only a part (e.g. because of security reasons) you can't just use the same dockerfile with the changed component and be sure that you'll get an otherwise identical system. So you don't get flexible updating. Cloning also required an already set-in-stone configured system to be the prototype. But until one reaches that, they have to experiment with various configurations (to end with the specific Docker image they want). Without reproducibility, they can't be guaranteed that their various test configurations under test are the same and only their latest changes differ. Perhaps another thing they left as is broke too. >I don't know of any big technology company that doesn't design their live infrastructure around ephemeral machines cloned from a template. That's just an argument from popularity. And not necessarily a popularity they opted for, while also being provided with the alternative. That they use "ephemeral machines cloned from a template" is just what they have to do -- not what would be ideal. Besides, reproducibility doesn't preclude cloning as part of the process -- it's a superset of it, offering way more flexibility. >I could simply create a new clone, get it working, then tag it as the next release. But if you mess with the clone it's not a clone anymore. And if a whole team has to mess with the clone over some period of time to update or fix it, there's hell to keep track what went on. Nobody argues that you can't simple install cloned images as they are.
      - geocar 10y ago
        > Only offering cloning means that starting with the same initial Dockerfile and making some needed updates to it, one can't be sure that the NON updated parts will be producible. So you don't get flexible reuse. What exactly do you mean by flexible reuse, and why is it good? What exactly do you mean by NON updated parts and what does it mean that they are "producible"? I don't understand your words. Speak plainly! I don't use a Dockerfile, and the interactive development approach means normally you start up your clone, work on it for a bit, then commit the clone. Here's an example: $ docker run -t -i debian:jessie bash root@264b51d62d0e:/# apt-get install nvi E: Unable to locate package nvi root@264b51d62d0e:/# apt-get update root@264b51d62d0e:/# apt-get install nvi $ v=`docker commit 264b51d62d0e` $ docker tag $v whatever The mistake with the first line didn't mean I needed to start over. In a realistic example that could save me an hour of rebuilding. I can push my tag to other developers I'm working with, and they can use `docker diff` to find out what the differences are. > But until one reaches that, they have to experiment with various configurations (to end with the specific Docker image they want). Without reproducibility, they can't be guaranteed that their various test configurations under test are the same and only their latest changes differ. Perhaps another thing they left as is broke too. This is a danger that requires discipline. Committing instances is very cheap, so it is good practice to commit often, and keep notes (commit messages). I think version control is common enough that most sysadmins know how to do this, and other packaging systems suffer from the exact same problem. However because it's interactive, that need to experiment is satisfied fully and the sysadmin/user gets to use all of their tools to develop the working prototypes. > But if you mess with the clone it's not a clone anymore. And if a whole team has to mess with the clone over some period of time to update or fix it, there's hell to keep track what went on. This is not true. With Docker you are encouraged to make many clones and branches and try different things out. The clones that are useful get tagged and forwarded to others. A sysadmin who isn't using version control has other problems! > Only offering cloning also means that when you need to update only a part (e.g. because of security reasons) you can't just use the same dockerfile with the changed component and be sure that you'll get an otherwise identical system. So you don't get flexible updating. I don't understand this complaint, but it mentions dockerfiles again. Let me be clear: I never use dockerfiles. If I want to update for security reasons (as I mentioned in my heartbleed example), then I will find it easier and faster to implement the fixes interactively. Having to write my recipe for nix-os and run it and wait while it runs is insecure because you are vulnerable longer. > Besides, reproducibility doesn't preclude cloning as part of the process -- it's a superset of it, offering way more flexibility. The non-interactive (and "reproducible") approach is slower, less secure, more prone to errors. I don't believe it can possibly be "more flexible" since the interactive approach can do everything the non-interactive approach can do and faster. I do admit that Docker's build artefacts use more disk space and more Internet bandwidth than plain text nix configuration artefacts, but this is not fundamental to interactive development, which is specifically what I'm advocating. > Any significant contribution you might have for software engineering in regards to that? > That's just an argument from popularity. And not necessarily a popularity they opted for, while also being provided with the alternative. Arguing that "nobody does it so it must be wrong" isn't any better than "just because everyone does it doesn't make it right". I really think you need to learn about something and make up your own mind, instead of repeating "Docker is Bad" because you read something like that on a blog. I also think you should really try to understand what I am suggesting before you go arguing against it.
- Keats 10y ago
  So let's say you are deploying 10 servers. Are you fine with all of them being different? Or to get out of computers, you have a drug where they say it cures cancer 100% of the time and no one can reproduce the results, it's fine for you?
  - geocar 10y ago
    Straw man. Reproducible in this case is talking about the process or steps needed to make a machine. You do not need to repeat the steps to produce the first machine to produce the second machine. You can simply clone the first machine, and Docker makes this very easy to do. That I can clone a machine does not mean that I can clone the interaction between a patient and their drugs, but if I could it would most certainly be preferable to trying to repeat the experiments with larger and larger populations.

by Keats 10y ago

geocar 10y ago

You are assuming "that reproducibility is good". I believe this assumption needs to be challenged.

coldtea 10y ago
Any significant contribution you might have for software engineering in regards to that? Aside from merely believing that. I can't even tell what you mean. If I want to rebuild my machine same as before, or deploy build 100s of servers at different points in time with the same setup, reproducibility is necessary. There's not even one argument against that. If you have any, please oblige us. Should deployments be a surprise?
- geocar 10y ago
  > If I want to rebuild my machine same as before, or deploy build 100s of servers at different points in time with the same setup, reproducibility is necessary. No it isn't. Reproducible is talking about the process or steps needed to make a machine. You do not need to repeat the steps to produce the first machine to produce the second machine. You can simply clone the first machine, and Docker makes this very easy to do. Keats begins his article with the statement offered, without justification: In this post I'm going to assume that reproducibility is good and necessary. Since I don't believe that, and in fact believe the opposite, I take issue with the entire article. Indeed, he goes on to construct a straw man comparing the exact opposite of what I propose -- a non-interactive "dockerfile" and not even a particularly good one, with a very well thought-out "purely functional" build system. > Any significant contribution you might have for software engineering in regards to that? I don't know of any big technology company that doesn't design their live infrastructure around ephemeral machines cloned from a template. They don't all use Docker, and they don't all use interactive development, however, but these two tools can make it easier for people who (for example) don't have Google's economy of scale. > Should deployments be a surprise? No they should not. Deployments take less time with my interactive method because you do not have to wait to "rebuild everything" which can take hours. Remember heartbleed? People using nixos had to rebuild every component that used OpenSSL, then test them, then find something else broke and repeat the process. Only once they had finished building their new process could they roll out this process to all machines, doing one more final rebuild. Even if done perfectly this has a minimum of two iterations. I could simply create a new clone, get it working, then tag it as the next release. New instances can then be cloned from the release, monitored, and the old instances destroyed. The entire process took under an hour for the entire fleet, including reading about heartbleed on HN. This is clearly almost as fast as running `apt-get dist-upgrade` on each of the live systems, but without any of the risk. The organisational complexity, however is admittedly much higher than "wash, wait, repeat", which is why most system administrators could not even begin to use live clones until virtualization and tooling had become much more commonplace. > I can't even tell what you mean. And yet you feel qualified to tell me I'm wrong.
  - coldtea 10y ago
    >Reproducible is talking about the process or steps needed to make a machine. You do not need to repeat the steps to produce the first machine to produce the second machine. You can simply clone the first machine, and Docker makes this very easy to do. Cloning is not the answer since it's all or nothing -- where reproducibility can be achieved piecemeal. Only offering cloning means that starting with the same initial Dockerfile and making some needed updates to it, one can't be sure that the NON updated parts will be producible. So you don't get flexible reuse. Only offering cloning also means that when you need to update only a part (e.g. because of security reasons) you can't just use the same dockerfile with the changed component and be sure that you'll get an otherwise identical system. So you don't get flexible updating. Cloning also required an already set-in-stone configured system to be the prototype. But until one reaches that, they have to experiment with various configurations (to end with the specific Docker image they want). Without reproducibility, they can't be guaranteed that their various test configurations under test are the same and only their latest changes differ. Perhaps another thing they left as is broke too. >I don't know of any big technology company that doesn't design their live infrastructure around ephemeral machines cloned from a template. That's just an argument from popularity. And not necessarily a popularity they opted for, while also being provided with the alternative. That they use "ephemeral machines cloned from a template" is just what they have to do -- not what would be ideal. Besides, reproducibility doesn't preclude cloning as part of the process -- it's a superset of it, offering way more flexibility. >I could simply create a new clone, get it working, then tag it as the next release. But if you mess with the clone it's not a clone anymore. And if a whole team has to mess with the clone over some period of time to update or fix it, there's hell to keep track what went on. Nobody argues that you can't simple install cloned images as they are.
    - geocar 10y ago
      > Only offering cloning means that starting with the same initial Dockerfile and making some needed updates to it, one can't be sure that the NON updated parts will be producible. So you don't get flexible reuse. What exactly do you mean by flexible reuse, and why is it good? What exactly do you mean by NON updated parts and what does it mean that they are "producible"? I don't understand your words. Speak plainly! I don't use a Dockerfile, and the interactive development approach means normally you start up your clone, work on it for a bit, then commit the clone. Here's an example: $ docker run -t -i debian:jessie bash root@264b51d62d0e:/# apt-get install nvi E: Unable to locate package nvi root@264b51d62d0e:/# apt-get update root@264b51d62d0e:/# apt-get install nvi $ v=`docker commit 264b51d62d0e` $ docker tag $v whatever The mistake with the first line didn't mean I needed to start over. In a realistic example that could save me an hour of rebuilding. I can push my tag to other developers I'm working with, and they can use `docker diff` to find out what the differences are. > But until one reaches that, they have to experiment with various configurations (to end with the specific Docker image they want). Without reproducibility, they can't be guaranteed that their various test configurations under test are the same and only their latest changes differ. Perhaps another thing they left as is broke too. This is a danger that requires discipline. Committing instances is very cheap, so it is good practice to commit often, and keep notes (commit messages). I think version control is common enough that most sysadmins know how to do this, and other packaging systems suffer from the exact same problem. However because it's interactive, that need to experiment is satisfied fully and the sysadmin/user gets to use all of their tools to develop the working prototypes. > But if you mess with the clone it's not a clone anymore. And if a whole team has to mess with the clone over some period of time to update or fix it, there's hell to keep track what went on. This is not true. With Docker you are encouraged to make many clones and branches and try different things out. The clones that are useful get tagged and forwarded to others. A sysadmin who isn't using version control has other problems! > Only offering cloning also means that when you need to update only a part (e.g. because of security reasons) you can't just use the same dockerfile with the changed component and be sure that you'll get an otherwise identical system. So you don't get flexible updating. I don't understand this complaint, but it mentions dockerfiles again. Let me be clear: I never use dockerfiles. If I want to update for security reasons (as I mentioned in my heartbleed example), then I will find it easier and faster to implement the fixes interactively. Having to write my recipe for nix-os and run it and wait while it runs is insecure because you are vulnerable longer. > Besides, reproducibility doesn't preclude cloning as part of the process -- it's a superset of it, offering way more flexibility. The non-interactive (and "reproducible") approach is slower, less secure, more prone to errors. I don't believe it can possibly be "more flexible" since the interactive approach can do everything the non-interactive approach can do and faster. I do admit that Docker's build artefacts use more disk space and more Internet bandwidth than plain text nix configuration artefacts, but this is not fundamental to interactive development, which is specifically what I'm advocating. > Any significant contribution you might have for software engineering in regards to that? > That's just an argument from popularity. And not necessarily a popularity they opted for, while also being provided with the alternative. Arguing that "nobody does it so it must be wrong" isn't any better than "just because everyone does it doesn't make it right". I really think you need to learn about something and make up your own mind, instead of repeating "Docker is Bad" because you read something like that on a blog. I also think you should really try to understand what I am suggesting before you go arguing against it.
Keats 10y ago
So let's say you are deploying 10 servers. Are you fine with all of them being different? Or to get out of computers, you have a drug where they say it cures cancer 100% of the time and no one can reproduce the results, it's fine for you?
- geocar 10y ago
  Straw man. Reproducible in this case is talking about the process or steps needed to make a machine. You do not need to repeat the steps to produce the first machine to produce the second machine. You can simply clone the first machine, and Docker makes this very easy to do. That I can clone a machine does not mean that I can clone the interaction between a patient and their drugs, but if I could it would most certainly be preferable to trying to repeat the experiments with larger and larger populations.