9 ms·

Show HN: Dockerfiler: declarative management of images built from Dockerfiles

35 points by jbergknoff 6y ago

leetrout 6y ago
So this is for me to easily manage all my linters, formatters and such in one image from a group of upstream, published images, right?
- pacifika 6y ago
  Looks like it. When developer tooling causes conflicts to install globally and tricky to configure with the project (say phpunit) then this can all be taken care of by building it within a docker container. However if the linting also dependent on a specific version of a programming language to test its code now you are no longer testing the production environment.
gavinray 6y ago
I think I may be a bit slow -- I don't get it =/
- joombaga 6y ago
  You write declarative manifests that include source repos/tags and destination repos/tags and build arguments, and this thing figures out what commands are needed to get your container registry in the desired state. It outputs those commands. You'd then pass then to an environment that has access rights to make the necessary changes.
  - gavinray 6y ago
    Ohhhh. So the example in the repo being, you have some set of tools which you may want to run in CI/CD, or maybe every developer to have the same version of (linters, deployment tools, test runners, etc) and this will figure out the optimal way of pulling them so they're available? Thanks, makes sense now I think.
    - joombaga 6y ago
      Yep! Not only pulling them, but also building them and pushing them if needed.
digitalsushi 6y ago
I am growing my anxiety as more layers to the Docker onion grow faster than I can stay current. In the past 2 years my company adopted kubernetes to manage our already confusing Docker infrastructure design, and then they added Rancher to it because no one was making sense of kubernetes. Meanwhile we're shipping on five year old containers that everyone is afraid to update and no one remembers how to build them. We're building a skyscraper on a floating pier and trying to reach the moon before the tide changes. I know this is the way, but I am having a hard time because it just doesn't make sense. This doesn't feel like process. It feels like compounding of reactions. Is it just me? My company? Or is this a general feeling?
- donut 6y ago
  > shipping on five year old containers that everyone is afraid to update and no one remembers how to build them. That's bad. This is not the way to do it. And honestly, this isn't the fault of Docker, Kubernetes, Rancher, or any tech. Sounds like the org is broken...
  - csunbird 6y ago
    I second this. This can also be applied to a jar/script that somebody built 5 years ago and nobody knows how to update it.
- ldoughty 6y ago
  If you can't rebuild the docker containers... I think you're doing things wrong... I always start from an official container base from a reputable source (Debian, Apache, nginx, Alpine)... Or I branch off one of my creations that is based on these... If I want to use someone else's work from an untrusted source, I make my own image and build pipeline for it so I'm in control. This is my philosophy... I don't have any containers I'm afraid to rebuild... But I don't use kubernetes or rancher, just raw docker, docker compose, and ansible/terraform
  - vladsanchez 6y ago
    I like and align with your simple approach: Docker Compose, Ansible and Terraform. K8s is too complicated for my competence and needs.
fermienrico 6y ago
We've made all this so complicated. Instead of the beautiful simplicity based on UNIX philosophy (arguably), modern devops feels like a mishmash of ideas plopped together, tied with a bunch of yaml tape and god forbid if you ever want to look inside the stack of mess. Containerization is a great idea but flawed in its interface. I like what Jim Keller (chip architect) says about complexity - that we need to throw away everything and start from scratch to which the interviewer asks, how often? Jim responds that currently chip architectures are re-done every 10 years but it should be more like every 5 years[1]. Just like any evolutionary process, there comes a point of diminishing returns because mistakes made cannot be corrected due to many other things that get piled up on top of it. So, it is difficult to track back. What happens is more shit gets piled up on top just to patch up old mistakes. Like our laryngeal nerve that loops around from the brain, all the way to the cervical area and goes back to the voicebox[2]. It is even more evident in a Giraffe. A good architect wouldn't design anatomy like this. The reason why it is the way it is, is because evolution has no hindsight and marginal cost of undoing the nerve is higher than just slightly increasing the length of the nerve. This is what we do in software. A good architect wouldn't design software like this. Sorry for the diversion, but I just feel so much pain with Docker, Kubernetes, Terraform and a whole load of AWS complexity. Holyshit. [1] https://www.youtube.com/watch?v=1CSeY10zbqo https://www.youtube.com/watch?v=1CSeY10zbqo [2] https://en.wikipedia.org/wiki/Recurrent_laryngeal_nerve#/media/File:Recurrent_laryngeal_nerve.svg https://en.wikipedia.org/wiki/Recurrent_laryngeal_nerve#/med...
- aasasd 6y ago
  > Instead of the beautiful simplicity based on UNIX philosophy, modern devops feels like a mishmash of ideas plopped together, tied with a bunch of yaml tape So... it's simpler tools tied together? Are you arguing that simple tools must instead stay apart? Is the developer supposed to build their thing invoking each basic tool one-by-one? Maybe makefiles and shell scripts should also be verboten, then?
  - aasasd 6y ago
    > we need to throw away everything and start from scratch Wait a minute, this part is even funnier. So Docker allows you to build your quasi-VMs using all those Unix tools, by talking to Linux that's been put inside your Linux. But apparently this is not what you meant by ‘Unix philosophy’ and it all should be thrown away. Huh.
techntoke 6y ago
From the article: > Docker is an excellent means of distributing those sorts of tools. No, package managers are excellent means of distributing and managing installed tools. Docker is an excellent way to package the tools, but it's distribution and management are terrible. There isn't even a command to simply show which containers are considered "outdated" without having to repull all your images, which can take 10+ minutes on 30 images.
jbergknoff 6y ago
Hi, I'm the author. Thanks for taking a look. This is basically a tool to help manage a "Dockerfile" repo (along the lines of https://github.com/jessfraz/dockerfiles https://github.com/jessfraz/dockerfiles), where you build any tools you want into images that you control. This can be really useful for personal use or within a company. Why build tools into Docker images? Love it or hate it, there are many senses in which Docker is currently the best medium that we have for distributing and running dev tools. Here's an article making that argument: https://jonathan.bergknoff.com/journal/run-more-stuff-in-docker/ https://jonathan.bergknoff.com/journal/run-more-stuff-in-doc....
- tuananh 6y ago
  this description is a lot clearer to me than the one from readme :)
  - jbergknoff 6y ago
    Thanks for that feedback! I'll update the readme.
aasasd 6y ago
Pretty sure the name is ungoogleable.
rsa25519 6y ago
See also Nix, which can declaratively generate Dockerfiles (example on home page) :-) https://nixos.org https://nixos.org
- TheDong 6y ago
  > declaratively generate Dockerfiles This is inaccurate, assuming you are referring to the 'dockerTools.buildLayeredImage' function in the nixpkgs repository. That builds docker images declaratively, not dockerfiles.
pacifika 6y ago
I guess we are fixing our lack of compatibility story by encasing our tools in fixed environments. Then write new tooling when this is no longer usable. That seems like a short term view of things.
techntoke 6y ago
That is what I use Skaffold for

35 points by jbergknoff 6y ago

So this is for me to easily manage all my linters, formatters and such in one image from a group of upstream, published images, right?

pacifika 6y ago
Looks like it. When developer tooling causes conflicts to install globally and tricky to configure with the project (say phpunit) then this can all be taken care of by building it within a docker container. However if the linting also dependent on a specific version of a programming language to test its code now you are no longer testing the production environment.

gavinray 6y ago

I think I may be a bit slow -- I don't get it =/

joombaga 6y ago
You write declarative manifests that include source repos/tags and destination repos/tags and build arguments, and this thing figures out what commands are needed to get your container registry in the desired state. It outputs those commands. You'd then pass then to an environment that has access rights to make the necessary changes.
- gavinray 6y ago
  Ohhhh. So the example in the repo being, you have some set of tools which you may want to run in CI/CD, or maybe every developer to have the same version of (linters, deployment tools, test runners, etc) and this will figure out the optimal way of pulling them so they're available? Thanks, makes sense now I think.
  - joombaga 6y ago
    Yep! Not only pulling them, but also building them and pushing them if needed.

digitalsushi 6y ago

I am growing my anxiety as more layers to the Docker onion grow faster than I can stay current. In the past 2 years my company adopted kubernetes to manage our already confusing Docker infrastructure design, and then they added Rancher to it because no one was making sense of kubernetes. Meanwhile we're shipping on five year old containers that everyone is afraid to update and no one remembers how to build them. We're building a skyscraper on a floating pier and trying to reach the moon before the tide changes. I know this is the way, but I am having a hard time because it just doesn't make sense. This doesn't feel like process. It feels like compounding of reactions. Is it just me? My company? Or is this a general feeling?

donut 6y ago
> shipping on five year old containers that everyone is afraid to update and no one remembers how to build them. That's bad. This is not the way to do it. And honestly, this isn't the fault of Docker, Kubernetes, Rancher, or any tech. Sounds like the org is broken...
- csunbird 6y ago
  I second this. This can also be applied to a jar/script that somebody built 5 years ago and nobody knows how to update it.
ldoughty 6y ago
If you can't rebuild the docker containers... I think you're doing things wrong... I always start from an official container base from a reputable source (Debian, Apache, nginx, Alpine)... Or I branch off one of my creations that is based on these... If I want to use someone else's work from an untrusted source, I make my own image and build pipeline for it so I'm in control. This is my philosophy... I don't have any containers I'm afraid to rebuild... But I don't use kubernetes or rancher, just raw docker, docker compose, and ansible/terraform
- vladsanchez 6y ago
  I like and align with your simple approach: Docker Compose, Ansible and Terraform. K8s is too complicated for my competence and needs.

fermienrico 6y ago

We've made all this so complicated. Instead of the beautiful simplicity based on UNIX philosophy (arguably), modern devops feels like a mishmash of ideas plopped together, tied with a bunch of yaml tape and god forbid if you ever want to look inside the stack of mess. Containerization is a great idea but flawed in its interface. I like what Jim Keller (chip architect) says about complexity - that we need to throw away everything and start from scratch to which the interviewer asks, how often? Jim responds that currently chip architectures are re-done every 10 years but it should be more like every 5 years[1]. Just like any evolutionary process, there comes a point of diminishing returns because mistakes made cannot be corrected due to many other things that get piled up on top of it. So, it is difficult to track back. What happens is more shit gets piled up on top just to patch up old mistakes. Like our laryngeal nerve that loops around from the brain, all the way to the cervical area and goes back to the voicebox[2]. It is even more evident in a Giraffe. A good architect wouldn't design anatomy like this. The reason why it is the way it is, is because evolution has no hindsight and marginal cost of undoing the nerve is higher than just slightly increasing the length of the nerve. This is what we do in software. A good architect wouldn't design software like this. Sorry for the diversion, but I just feel so much pain with Docker, Kubernetes, Terraform and a whole load of AWS complexity. Holyshit. [1] https://www.youtube.com/watch?v=1CSeY10zbqo https://www.youtube.com/watch?v=1CSeY10zbqo [2] https://en.wikipedia.org/wiki/Recurrent_laryngeal_nerve#/media/File:Recurrent_laryngeal_nerve.svg https://en.wikipedia.org/wiki/Recurrent_laryngeal_nerve#/med...

aasasd 6y ago
> Instead of the beautiful simplicity based on UNIX philosophy, modern devops feels like a mishmash of ideas plopped together, tied with a bunch of yaml tape So... it's simpler tools tied together? Are you arguing that simple tools must instead stay apart? Is the developer supposed to build their thing invoking each basic tool one-by-one? Maybe makefiles and shell scripts should also be verboten, then?
- aasasd 6y ago
  > we need to throw away everything and start from scratch Wait a minute, this part is even funnier. So Docker allows you to build your quasi-VMs using all those Unix tools, by talking to Linux that's been put inside your Linux. But apparently this is not what you meant by ‘Unix philosophy’ and it all should be thrown away. Huh.

techntoke 6y ago

From the article: > Docker is an excellent means of distributing those sorts of tools. No, package managers are excellent means of distributing and managing installed tools. Docker is an excellent way to package the tools, but it's distribution and management are terrible. There isn't even a command to simply show which containers are considered "outdated" without having to repull all your images, which can take 10+ minutes on 30 images.

jbergknoff 6y ago

Hi, I'm the author. Thanks for taking a look. This is basically a tool to help manage a "Dockerfile" repo (along the lines of https://github.com/jessfraz/dockerfiles https://github.com/jessfraz/dockerfiles), where you build any tools you want into images that you control. This can be really useful for personal use or within a company. Why build tools into Docker images? Love it or hate it, there are many senses in which Docker is currently the best medium that we have for distributing and running dev tools. Here's an article making that argument: https://jonathan.bergknoff.com/journal/run-more-stuff-in-docker/ https://jonathan.bergknoff.com/journal/run-more-stuff-in-doc....

tuananh 6y ago
this description is a lot clearer to me than the one from readme :)
- jbergknoff 6y ago
  Thanks for that feedback! I'll update the readme.

aasasd 6y ago

Pretty sure the name is ungoogleable.

rsa25519 6y ago

See also Nix, which can declaratively generate Dockerfiles (example on home page) :-) https://nixos.org https://nixos.org

TheDong 6y ago
> declaratively generate Dockerfiles This is inaccurate, assuming you are referring to the 'dockerTools.buildLayeredImage' function in the nixpkgs repository. That builds docker images declaratively, not dockerfiles.

pacifika 6y ago

I guess we are fixing our lack of compatibility story by encasing our tools in fixed environments. Then write new tooling when this is no longer usable. That seems like a short term view of things.

techntoke 6y ago

That is what I use Skaffold for