6 ms·

For me, docker is just a superpower that let's me build larger, more complex applications. That's why it's worth learning. It raises the ceiling of what I'm abl

by cdrini 6y ago

For me, docker is just a superpower that let's me build larger, more complex applications. That's why it's worth learning. It raises the ceiling of what I'm able to create. Creating a site with multiple services that integrate with each other is complicated. You could do it from scratch, but it would take so long that you would likely never be able to manage the actual complexity you care about--the complexity you want to architect and program. You'd be too busy programming solutions to the problems that docker already solves.

SilverRed 6y ago
I love docker as a home user as well. Sometimes when I'm bored I'll build an IM bot. Docker makes hosting it super simple. I just start from the ruby docker image, copy my script in and then dump it on gitlab which will build and host the container which lets me pull it from my server. I can then copy a simple systemd config to start it on boot and restart it if it fails. This is all much simpler than managing multiple versions of ruby installed locally. Not to mention the security benefits of keeping it in a container. Perhaps this is all so easy and convenient for me because I learned docker for pro use and now its just mentally free for casual use.
- vincnetas 6y ago
  Docker has autorestart on failure. I think it also autorestarts after reboot.
  - SilverRed 6y ago
    Yeah I have seen this but for some reason it just wasn't working for me. I think that maybe the docker service just wasn't being started at boot but once I set it up with systemd it all just works now.
    - lostlogin 6y ago
      I find it painfully reliable. I forget about it and then weeks later I find the image when scrounging about trying to work out where my RAM went.
latch 6y ago
> You could do it from scratch, but it would take so long that you would likely never be able to manage the actual complexity you care about Except plenty of people have and continue to manage? Do we think that _most_ people are using Docker now? This is how we do it: We use baremetal servers. We use systemd, including the systemd sandboxing capabilities. We have deploy scripts (almost all bash, and yes, they are largely idempotent). A host could be deployed with any number of: cockroachdb, openresty, monetdb, rabbitmq or postgresql. They're all running vector (log ingestion). And they can be running N apps. Each host also get public keys for encrypting on-site, credentials for uploading backups to Google Cloud, systemd-timers for tasks like running lynis and rkhunter. Everything is connected via wireguard (over a physical private network). Our apps (mostly go and elixir) are built on a self-hosted gitlab and we deploy like normal. Each project can pick what it wants, e.g. auto deploy dev branch to dev, auto deploy master to stage, manually-triggered to prod. We run separate tool servers for our internal stuff, gitlab, elastic search, kibana, prometheus, vault and grafana. We have some clustering/mesh networking which we "discover" with an internal DNS. But everything is pretty static/fixed. We can survive downtime because either something is HA and stateless (e.g. we keep N monetdb "reporting" servers in sync from the OLTP->OLAP process), or stateful and part of a cluster (e.g., cockroachdb). Here's the kicker: we have 0 devops. We don't spend a huge deal of time on this. We used to have devops and K8 and everything was worse. It was _much_ more expensive, everything took _a lot_ longer to get done, it all ran slower, it was much buggier, and it was less secure (because we aren't sharing a host). I feel like we could build almost anything this way. I don't see what complexity we'd run into that Docker would help with. I think we'd run into management issues...applying patches to hundreds of servers, reviewing hardening and access logs, etc. But that's infrastructure complexity, not application complexity. We use docker in apps that require a complex setup for end-to-end testing.
- sofixa 6y ago
  What you have sounds great, and sounds about the same as running containers with orchestration, just via SystemD cgroups instead of container cgroups, and bash and humans instead of a real orchestrator. You lose a lot of features ( idempotent and self-contained ( so zero risk of dependency clashing), cluster-wide auto-healing, load distribution and autoscaling, centralised management) and ease of use of existing solutions ( want to run a CockroachDB cluster? There's probably a k8s operator for that; want automatic DNS? Consul from Nomad, k8s already do that), but you understand it. There could be very serious downsides around maintenance of your bash scripts, and onboarding new people.

by cdrini 6y ago

SilverRed 6y ago

I love docker as a home user as well. Sometimes when I'm bored I'll build an IM bot. Docker makes hosting it super simple. I just start from the ruby docker image, copy my script in and then dump it on gitlab which will build and host the container which lets me pull it from my server. I can then copy a simple systemd config to start it on boot and restart it if it fails. This is all much simpler than managing multiple versions of ruby installed locally. Not to mention the security benefits of keeping it in a container. Perhaps this is all so easy and convenient for me because I learned docker for pro use and now its just mentally free for casual use.

vincnetas 6y ago
Docker has autorestart on failure. I think it also autorestarts after reboot.
- SilverRed 6y ago
  Yeah I have seen this but for some reason it just wasn't working for me. I think that maybe the docker service just wasn't being started at boot but once I set it up with systemd it all just works now.
  - lostlogin 6y ago
    I find it painfully reliable. I forget about it and then weeks later I find the image when scrounging about trying to work out where my RAM went.

latch 6y ago

> You could do it from scratch, but it would take so long that you would likely never be able to manage the actual complexity you care about Except plenty of people have and continue to manage? Do we think that _most_ people are using Docker now? This is how we do it: We use baremetal servers. We use systemd, including the systemd sandboxing capabilities. We have deploy scripts (almost all bash, and yes, they are largely idempotent). A host could be deployed with any number of: cockroachdb, openresty, monetdb, rabbitmq or postgresql. They're all running vector (log ingestion). And they can be running N apps. Each host also get public keys for encrypting on-site, credentials for uploading backups to Google Cloud, systemd-timers for tasks like running lynis and rkhunter. Everything is connected via wireguard (over a physical private network). Our apps (mostly go and elixir) are built on a self-hosted gitlab and we deploy like normal. Each project can pick what it wants, e.g. auto deploy dev branch to dev, auto deploy master to stage, manually-triggered to prod. We run separate tool servers for our internal stuff, gitlab, elastic search, kibana, prometheus, vault and grafana. We have some clustering/mesh networking which we "discover" with an internal DNS. But everything is pretty static/fixed. We can survive downtime because either something is HA and stateless (e.g. we keep N monetdb "reporting" servers in sync from the OLTP->OLAP process), or stateful and part of a cluster (e.g., cockroachdb). Here's the kicker: we have 0 devops. We don't spend a huge deal of time on this. We used to have devops and K8 and everything was worse. It was _much_ more expensive, everything took _a lot_ longer to get done, it all ran slower, it was much buggier, and it was less secure (because we aren't sharing a host). I feel like we could build almost anything this way. I don't see what complexity we'd run into that Docker would help with. I think we'd run into management issues...applying patches to hundreds of servers, reviewing hardening and access logs, etc. But that's infrastructure complexity, not application complexity. We use docker in apps that require a complex setup for end-to-end testing.

sofixa 6y ago
What you have sounds great, and sounds about the same as running containers with orchestration, just via SystemD cgroups instead of container cgroups, and bash and humans instead of a real orchestrator. You lose a lot of features ( idempotent and self-contained ( so zero risk of dependency clashing), cluster-wide auto-healing, load distribution and autoscaling, centralised management) and ease of use of existing solutions ( want to run a CockroachDB cluster? There's probably a k8s operator for that; want automatic DNS? Consul from Nomad, k8s already do that), but you understand it. There could be very serious downsides around maintenance of your bash scripts, and onboarding new people.