3 ms·
> You could do it from scratch, but it would take so long that you would likely never be able to manage the actual complexity you care about Except plenty of p
by latch 6y ago
> You could do it from scratch, but it would take so long that you would likely never be able to manage the actual complexity you care about
Except plenty of people have and continue to manage? Do we think that _most_ people are using Docker now?
This is how we do it:
We use baremetal servers. We use systemd, including the systemd sandboxing capabilities. We have deploy scripts (almost all bash, and yes, they are largely idempotent). A host could be deployed with any number of: cockroachdb, openresty, monetdb, rabbitmq or postgresql. They're all running vector (log ingestion). And they can be running N apps. Each host also get public keys for encrypting on-site, credentials for uploading backups to Google Cloud, systemd-timers for tasks like running lynis and rkhunter. Everything is connected via wireguard (over a physical private network).
Our apps (mostly go and elixir) are built on a self-hosted gitlab and we deploy like normal. Each project can pick what it wants, e.g. auto deploy dev branch to dev, auto deploy master to stage, manually-triggered to prod.
We run separate tool servers for our internal stuff, gitlab, elastic search, kibana, prometheus, vault and grafana.
We have some clustering/mesh networking which we "discover" with an internal DNS. But everything is pretty static/fixed. We can survive downtime because either something is HA and stateless (e.g. we keep N monetdb "reporting" servers in sync from the OLTP->OLAP process), or stateful and part of a cluster (e.g., cockroachdb).
Here's the kicker: we have 0 devops. We don't spend a huge deal of time on this. We used to have devops and K8 and everything was worse. It was _much_ more expensive, everything took _a lot_ longer to get done, it all ran slower, it was much buggier, and it was less secure (because we aren't sharing a host).
I feel like we could build almost anything this way. I don't see what complexity we'd run into that Docker would help with. I think we'd run into management issues...applying patches to hundreds of servers, reviewing hardening and access logs, etc. But that's infrastructure complexity, not application complexity.
We use docker in apps that require a complex setup for end-to-end testing.
- sofixa 6y agoWhat you have sounds great, and sounds about the same as running containers with orchestration, just via SystemD cgroups instead of container cgroups, and bash and humans instead of a real orchestrator. You lose a lot of features ( idempotent and self-contained ( so zero risk of dependency clashing), cluster-wide auto-healing, load distribution and autoscaling, centralised management) and ease of use of existing solutions ( want to run a CockroachDB cluster? There's probably a k8s operator for that; want automatic DNS? Consul from Nomad, k8s already do that), but you understand it. There could be very serious downsides around maintenance of your bash scripts, and onboarding new people.
- pooper 6y agoPersonally, I don't want to think about all this complexity. I just build simple web applications. I want a box in my basement that has everything production has - the whole thing, all the web services running all the time. It ought to be possible in my mind. After all, there is really only one user: me. I just want to be able to automatically get the latest code that reservations team or order management system team or what have you deploys to production. Why do I need to connect to VPN just to work on a stupid web application? None of the code we write has any special sauce in it and we can have fake data...
- EdwardDiego 6y ago> largely idempotent Got to say, that "largely" scares me when discussing deployments.