4 ms·
From experience: Plain docker - hell on earth. Literally some of the worst stuff I had to deal with. A noticeable worsening vs. running the contents of the con
by p_l 7y ago
From experience:
Plain docker - hell on earth. Literally some of the worst stuff I had to deal with. A noticeable worsening vs. running the contents of the container unpacked in /opt/myapp.
Heroku, Dokku - Very depends. A dance between "Simple and works" and "Simple and Works but my Startup is bankrupt".
K8s - Do I have more than 1-2 custom deployed applications? Predictable cost model, simple enough to manage (granted I might be an aberration), easy to combine infrastructural parts with the money-making parts. Strong contender vs Heroku-like, especially on classic startup budget
- alexbanks 7y agoI have desperately avoided K8s since it became popular, but this seems to be pretty opposite of everything else I've read? Can you share what specifically makes plain docker so terrible?
- p_l 7y agoPlain docker (no swarm etc.) for practical purposes resulted in containers that were in weird position when it comes to decoupling from the host OS. Integration with init system was abysmal. Docker daemon had its own conventions it wanted to follow. Unless you ensure that the state of the docker daemon is deleted on reboot, you could have weird conditions when it tried to handle starting the containers by itself. A very easy thing to use for developer, a pretty shitty tool (assuming no external wrappers) on server. One of the greatest joys of k8s for me was always "it abstracts docker away so I don't have to deal with it, and it drops pretty much all broken features of docker"
- mdaniel 7y ago> One of the greatest joys of k8s for me was always "it abstracts docker away so I don't have to deal with it, and it drops pretty much all broken features of docker" It also offers portability away from docker via the Container Runtime Interface; we use containerd and it has been absolutely rock solid, without the weird "what happens to my containers if I have to restart a wedged dockerd?" situation
- p_l 7y agoIndeed - Back when the two alternatives were very alpha quality rktnetes and hypernetes, I was already full of joy that I was seeing the end of docker on server. Since then we got CRI-O and life looks even better.
- simplecto 7y agoI always start and keep my containers stateless. the docker run --rm command line switch tell docker to remove the container when it dies. Never a problem on restarts.
- pnako 7y agoYou've decoupled from the host OS (good thinking), and you've abstracted away from Docker (even better thinking), but how did you decouple from k8s?
- Spivak 7y agoIf you are a user of k8s then the experience relatively smooth and painless since all the complexity is hidden behind the API you're calling to provision resources for you. If you are an operator or k8s then you've entered the nightmare zone where you have to make all of those endpoints actually work and do the right thing with code that written 17 days ago. Unlimited terrible middleware to try and form static services into dynamic boxes. k8s was not designed for someone to deploy on-prem that doesn't have a dedicated team of developers and ops people to work on just k8s.
- alexbanks 7y agoSince you've made this distinction, can you explain more? How would the person I replied to not be both? In what scenario are they different people?
- p_l 7y agoI mostly use it, but I also tend to keep to stable features so that running on-prem is also relatively simple. My biggest cryparty story when it comes to on-prem kubernetes is not actually due to kubernetes, but due to Red Hat. There are words I could say about their official OpenShift deployment scripts and their authors, but I would be rightly banned for writing them. Biggest issues I've encountered involve things on the interface between "classic distro" and running k8s on top of it, and that goes down when you move towards base OS that is more optimized for k8s (for example flatcar). When it comes to the size of team involved, I'd say keeping "classic" stack with loadbalancers, pacemaker, custom deployment methods etc. was comparable effort - at least if we're matching feature for feature what I'd see on "base" k8s on-prem setup (and based on "what I wish I had, or that I could replace with k8s, back in 2016 for an on-prem project). There's one thing, however, where it gets much harder to deploy k8s and I won't hide it - when you're dealing with random broken classic infrastructure with no authority to change it. K8s gets noticeably harder when you need to deal with annoying pre-existing IP layouts because you can't get IP address space allocated, when you have to sit on badly stretched L2 domains, where the local idea of routing is OSPF - or worse, static routes to gateway and a stretched L2. To the point that sometimes it's easier to setup a base "private cloud" (I like VMware for this) on the physical machines, then setup the hosts on top of that - even if you're only going to have one VM per hypervisor. The benefits of abstracting possibly broken infrastructure are too big.
- golergka 7y agoYou seem like you know what you're talking about. What do you think about AWS RDS for db and EBS for auto-scaling and deployment? (It's my first fullstack product and it feels as if it's both a good developer experience and something that won't cost as an arm and a leg, so I just have to ask).
- p_l 7y agoAWS RDS is pretty nice, and 95% of the time you won't even notice any problems (it is, however, pretty complex like all of AWS and there are ways to wedge yourself into weird condition). Generally, if you don't have specific and well reasoned motivation to deploy a database (assuming your stack is based on DB supported by RDS) on your own, and you're running in AWS, it's a no brainer. As for EBS, remember that the SLAs for EBS are not the same as for S3, and that EBS volumes can be surprisingly slow (especially in IOPS terms once you go above certain limit, I don't have the numbers in cache at the moment). So it's important to have good backup/recovery/resiliency plan for anything deployed on EC2 or dependant on EBS volumes. Planning for speed is mostly a case when you do need a more custom datastore than offerred by AWS. Remember that AWS definitely prefers applications that go all vendor lock-in for various higher-level options from them, or ones that are at least "cloud native". Replicating an on-prem setup usually ends up in large bills and lower availability for little to no gain.
- simplecto 7y agoI run plain docker without issue.