7 ms·

people dramatically overestimate how difficult it is to write a program that controls docker for you. This is one of those things where you can write like two p

by zemo 2y ago

people dramatically overestimate how difficult it is to write a program that controls docker for you. This is one of those things where you can write like two pages of Python and ignore... all this: > Tealok is a runtime we’re building for running containers. If you have one machine and docker-compose is falling short, really, just write a Python script with the official docker Python package, you'll be fine.

SomeCallMeTim 2y ago
Ha. Just did almost exactly that, but with a Go script--I wanted my Docker Compose to auto-update when I built on my CI server. I found Watchtower, but polling just struck me as the Wrong Answer. Both too much overhead to keep pinging for the latest builds, and too slow to actually download the latest build. So I took some hints from Watchtower as to what I needed to do (mount the docker sock as a volume) and wrote a tiny Go server that, when pinged with a shared secret, would cause it to run `docker compose up -d --pull always`. Probably took me an hour. Then I added the ability to purge images before each update, because my tiny VM kept running out of disk space in the Docker partition. Oops. Scripts FTW. I was already using the suggestion in the article about having a single reverse proxy server to redirect different paths (and different domains) to different servers hosted in the Compose file. Seemed like the obvious answer. And I've configured k8s for my day job, so I could be using that. But I'm using Compose because I know how much of a pain k8s can be, especially around upgrades, where it has a habit of deprecating older versions of various interfaces. I'll put in the work for someone who's paying me to do it, but I'd rather work on my side project, not on configuring k8s.
- morcus 2y ago
  > I found Watchtower, but polling just struck me as the Wrong Answer. Both too much overhead to keep pinging for the latest builds, and too slow to actually download the latest build. So I took some hints from Watchtower as to what I needed to do (mount the docker sock as a volume) and wrote a tiny Go server that, when pinged with a shared secret, would cause it to run `docker compose up -d --pull always`. Is the code for this public? I had the same desire when setting up the server for my personal project but as you mentioned I eventually decided it was OK to just poll every two minutes and I would rather work on the project than on configuring docker. What I would like though is cleaning up the older images that are no longer needed.
chefandy 2y ago
I'm not dev-ops-familiar with Docker, so you might be more familiar with the problem space, but it seems like "You can just write a Python script to do what you want" is the sort of thing people say to justify not giving people the config infrastructure they need to solve their problems without having to write executable code. Like, sure, you often can make up for not having a zoom lens by walking towards what you're photographing, but that doesn't mean zoom lenses aren't useful to most, and critical to some.
- hamandcheese 2y ago
  > to solve their problems without having to write executable code. Some people would rather write 1000 lines of yaml than 100 lines of python, and I really don't understand why.
  - chefandy 2y ago
    It brings a new layer of complexity, which means more surface area for bugs and vulnerabilities. It’s often easier for developers to write a script using standardized tooling than dig into deep configuration docs for a complex application you’re not familiar with, but that’s where the benefits stop. Configuring built-in functionality makes sense for the same reason using an existing framework/os/etc authentication system makes sense. It often seems like way more effort to learn how to use it than rolling your own simple system, but most of that complexity deals with edge cases you haven’t been bitten by yet. Your implementation doesn’t need to get very big or last very long before the tail of your pipeline logic gets unnecessarily long. Those features don’t exist merely because some people are scared of code. Additionally, if you’re just telling the application to do something it already does, writing code using general purpose tools will almost certainly be more verbose. Even if not, 10 to 1 is fantastically hyperbolic. And unless you write code for a living— and many dev ops people do not— defining a bunch of boolean and string values to control heavily tested, built-in application functionality (that you should understand anyway before making a production deployment) requires way less mental overhead than writing secure, reliable, production-safe pipeline code that will then need to be debugged and maintained as a separate application when anything it touches gets changed. Updating configuration options is much simpler than figuring out how to deal with application updates in code, and the process is probably documented in release notes so you’re much more likely to realize you need to change something before stuff breaks.
nicce 2y ago
> If you have one machine and docker-compose is falling short, really, just write a Python script with the official docker Python package, you'll be fine. … aand there is even Ansible playbook for that! Maybe overkill.
lolinder 2y ago
It's more than just a runtime for running containers, from the main landing page it looks like they're really specifically targeting self-hosting for the barely-technical [0]. In that context this article makes some degree of sense: their target audience will probably be legitimately overwhelmed by that pihole example and wouldn't know how to write a Python script to save their life. If they can pull it off more power to them, but I'm skeptical that what the non-technical self-hoster needs is a TOML DSL that abstracts away ports, volumes, and even database implementations behind a magic text file. At some point you just have to bite the bullet and admit that your audience needs a GUI (like Sandstorm [1]). [0] https://tealok.tech/ https://tealok.tech/ [1] https://sandstorm.io/ https://sandstorm.io/
- eliribble 2y ago
  I'm actually a fan of Sandstorm, and think it got a lot of things right. I'd love to be able to talk to Kenton Varda about why he thinks adoption on it was weak. Personally I think that it put a bit too much burden on application developers since it required them to develop applications specifically for sandstorm. > I'm skeptical that what the non-technical self-hoster needs is a TOML DSL that abstracts away ports I fully agree, the end user would not be writing TOML DSL files. The end user would get something much closer to an app store, or what Sandstorm did, with one (or a few) click installs. The TOML DSL would be written by developers familiar with the application and stored either in a separate database, or ideally in the applications source control like a Dockerfile.
  - kentonv 2y ago
    > I'd love to be able to talk to Kenton Varda about why he thinks adoption on it was weak. Oh hai. Honestly I'm not sure I'm a reliable source for why we failed. It's tempting to convince myself of convenient excuses. But I really don't think the problem was with the idea. We actually had a lot of user excitement around the product. I think we screwed up the business strategy. We were too eager to generate revenue too early on, and that led us to focus efforts in the wrong areas, away from the things that would have been best for long-term growth. And we were totally clueless about enterprise sales, but didn't realize how clueless we were until it was too late (a classic blunder). Investors really don't like it when you say you're going to try for revenue and then you don't, so we were pretty much dead at that point.
tommica 2y ago
Do you have some reference that you can link to? Would be curious to see something about this
- echoangle 2y ago
  Here’s the python module you would use: https://docker-py.readthedocs.io/en/stable/ https://docker-py.readthedocs.io/en/stable/
lokar 2y ago
IME, people dramatically overestimate how hard it is to write any program.
- bunderbunder 2y ago
  IME people overestimate how hard it should be to write a program, but underestimate how hard it actually is after they're done overengineering the problem to death.

by zemo 2y ago

SomeCallMeTim 2y ago

Ha. Just did almost exactly that, but with a Go script--I wanted my Docker Compose to auto-update when I built on my CI server. I found Watchtower, but polling just struck me as the Wrong Answer. Both too much overhead to keep pinging for the latest builds, and too slow to actually download the latest build. So I took some hints from Watchtower as to what I needed to do (mount the docker sock as a volume) and wrote a tiny Go server that, when pinged with a shared secret, would cause it to run `docker compose up -d --pull always`. Probably took me an hour. Then I added the ability to purge images before each update, because my tiny VM kept running out of disk space in the Docker partition. Oops. Scripts FTW. I was already using the suggestion in the article about having a single reverse proxy server to redirect different paths (and different domains) to different servers hosted in the Compose file. Seemed like the obvious answer. And I've configured k8s for my day job, so I could be using that. But I'm using Compose because I know how much of a pain k8s can be, especially around upgrades, where it has a habit of deprecating older versions of various interfaces. I'll put in the work for someone who's paying me to do it, but I'd rather work on my side project, not on configuring k8s.

morcus 2y ago
> I found Watchtower, but polling just struck me as the Wrong Answer. Both too much overhead to keep pinging for the latest builds, and too slow to actually download the latest build. So I took some hints from Watchtower as to what I needed to do (mount the docker sock as a volume) and wrote a tiny Go server that, when pinged with a shared secret, would cause it to run `docker compose up -d --pull always`. Is the code for this public? I had the same desire when setting up the server for my personal project but as you mentioned I eventually decided it was OK to just poll every two minutes and I would rather work on the project than on configuring docker. What I would like though is cleaning up the older images that are no longer needed.

chefandy 2y ago

I'm not dev-ops-familiar with Docker, so you might be more familiar with the problem space, but it seems like "You can just write a Python script to do what you want" is the sort of thing people say to justify not giving people the config infrastructure they need to solve their problems without having to write executable code. Like, sure, you often can make up for not having a zoom lens by walking towards what you're photographing, but that doesn't mean zoom lenses aren't useful to most, and critical to some.

hamandcheese 2y ago
> to solve their problems without having to write executable code. Some people would rather write 1000 lines of yaml than 100 lines of python, and I really don't understand why.
- chefandy 2y ago
  It brings a new layer of complexity, which means more surface area for bugs and vulnerabilities. It’s often easier for developers to write a script using standardized tooling than dig into deep configuration docs for a complex application you’re not familiar with, but that’s where the benefits stop. Configuring built-in functionality makes sense for the same reason using an existing framework/os/etc authentication system makes sense. It often seems like way more effort to learn how to use it than rolling your own simple system, but most of that complexity deals with edge cases you haven’t been bitten by yet. Your implementation doesn’t need to get very big or last very long before the tail of your pipeline logic gets unnecessarily long. Those features don’t exist merely because some people are scared of code. Additionally, if you’re just telling the application to do something it already does, writing code using general purpose tools will almost certainly be more verbose. Even if not, 10 to 1 is fantastically hyperbolic. And unless you write code for a living— and many dev ops people do not— defining a bunch of boolean and string values to control heavily tested, built-in application functionality (that you should understand anyway before making a production deployment) requires way less mental overhead than writing secure, reliable, production-safe pipeline code that will then need to be debugged and maintained as a separate application when anything it touches gets changed. Updating configuration options is much simpler than figuring out how to deal with application updates in code, and the process is probably documented in release notes so you’re much more likely to realize you need to change something before stuff breaks.

nicce 2y ago

> If you have one machine and docker-compose is falling short, really, just write a Python script with the official docker Python package, you'll be fine. … aand there is even Ansible playbook for that! Maybe overkill.

lolinder 2y ago

It's more than just a runtime for running containers, from the main landing page it looks like they're really specifically targeting self-hosting for the barely-technical [0]. In that context this article makes some degree of sense: their target audience will probably be legitimately overwhelmed by that pihole example and wouldn't know how to write a Python script to save their life. If they can pull it off more power to them, but I'm skeptical that what the non-technical self-hoster needs is a TOML DSL that abstracts away ports, volumes, and even database implementations behind a magic text file. At some point you just have to bite the bullet and admit that your audience needs a GUI (like Sandstorm [1]). [0] https://tealok.tech/ https://tealok.tech/ [1] https://sandstorm.io/ https://sandstorm.io/

eliribble 2y ago
I'm actually a fan of Sandstorm, and think it got a lot of things right. I'd love to be able to talk to Kenton Varda about why he thinks adoption on it was weak. Personally I think that it put a bit too much burden on application developers since it required them to develop applications specifically for sandstorm. > I'm skeptical that what the non-technical self-hoster needs is a TOML DSL that abstracts away ports I fully agree, the end user would not be writing TOML DSL files. The end user would get something much closer to an app store, or what Sandstorm did, with one (or a few) click installs. The TOML DSL would be written by developers familiar with the application and stored either in a separate database, or ideally in the applications source control like a Dockerfile.
- kentonv 2y ago
  > I'd love to be able to talk to Kenton Varda about why he thinks adoption on it was weak. Oh hai. Honestly I'm not sure I'm a reliable source for why we failed. It's tempting to convince myself of convenient excuses. But I really don't think the problem was with the idea. We actually had a lot of user excitement around the product. I think we screwed up the business strategy. We were too eager to generate revenue too early on, and that led us to focus efforts in the wrong areas, away from the things that would have been best for long-term growth. And we were totally clueless about enterprise sales, but didn't realize how clueless we were until it was too late (a classic blunder). Investors really don't like it when you say you're going to try for revenue and then you don't, so we were pretty much dead at that point.

tommica 2y ago

Do you have some reference that you can link to? Would be curious to see something about this

echoangle 2y ago
Here’s the python module you would use: https://docker-py.readthedocs.io/en/stable/ https://docker-py.readthedocs.io/en/stable/

lokar 2y ago

IME, people dramatically overestimate how hard it is to write any program.

bunderbunder 2y ago
IME people overestimate how hard it should be to write a program, but underestimate how hard it actually is after they're done overengineering the problem to death.