19 ms·

I don't get why people use Docker for "small SaaS apps" where "scale is not an issue". You can run everything without Docker to remove an extra level of indirec

• leetrout 6y ago
  Docker on Mac and Windows is so slow when used for a development environment but there are enough differences that it makes it worth using. It’s more portable than VMs (vagrant and such) and if you’re using your CI pipeline to build an image then deployment is a snap. I much prefer building a Docker image and pushing it somewhere compared to tarballs of the repo or repo access. I would rather build rpms or debs than just push the repo around. Container tooling makes that kind of stuff nice in my opinion.
  • mherrmann 6y ago
    You agree that Docker is "so slow" and point to its benefits for preventing differences between dev and prod. But you don't mention what those differences are. What are they? I also don't see why you'd use a CI pipeline for simple projects where you are typically the only developer. Run tests locally, if everything is fine push to prod. I use git with a small script to roll out to production. When I execute the script on the server, it stops all services, pulls the latest code from Git, applies Django migrations, and starts everything up again. I use desk [1] to make this as simple as a single `release` command in my local shell. [1]: https://github.com/jamesob/desk https://github.com/jamesob/desk
• ahmedalsudani 6y ago
  Allow me to offer a counterpoint. Docker is useful for reasons besides scale. It allows you to (kind of) declaratively define your environment and spin it up in any number of different scenarios. It allows you to ship all your dependencies with your app and not worry about getting the host machine properly configured beyond setting up Docker and/or Kubernetes. Have other apps you want to host on the same set of machines? No problem. When you you pair that with something like alpine Linux, you’re getting a whole lot almost for free.
  • mherrmann 6y ago
    How many scenarios do you typically need to spin up your environment in? I achieve the same (defining the environment) by pinning Python dependencies via requirements.txt and virtual environments. And I have an installation script, like a Dockerfile but just an executable bash script, that installs PostgreSQL etc, pulls the code from GitHub and starts up the server. I can upload this to a clean Debian installation to set up the server, with a well-defined stack, in minutes. > Have other apps you want to host on the same set of machines I don't. I just use one $5 per month Linode for each SaaS. (Or bigger Linodes as the projects get more users. My biggest single box currently serves ~100,000 users.)
    • piaste 6y ago
      > I don't. I just use one $5 per month Linode for each SaaS Well then you're basically using VMs as your containerization mechanism, with install scripts replacing Dockerfiles. So from your perspective, don't think of Docker as a really fat binary. Think of it as a stripped-down VM that doesn't cost a minimum of $5 per instance, and comes in a standardized format with a standardized ecosystem around it (package registries, monitoring dashboards, etc.)
      • mherrmann 6y ago
        It's a really fat binary that adds unnecessary layers in both dev and prod. In dev, it's faster to run tests and easier to debug without Docker. And in prod, why use a vm inside a vm?
• GiorgioG 6y ago
  If you know Docker, and use it daily - it’s a no brainer. For me it’s an easy choice - I dev on a Desktop PC but when I’m on the go I’m on a MacBook. No need to customize environment config with docker. What’s your disaster recovery plan if your server tips over? With docker it’s relatively easy. I’m not saying it’s for everyone, but in my experience it makes my life simpler.
  • zdkl 6y ago
    >What’s your disaster recovery plan if your server tips over? Git pull, python3 setup_prod.py, pip install requirements.txt Is docker worth not writing the 50 (admittedly unfun) lines of setup?
    • adgud 6y ago
      And you probably have rest of the stuff described in an Ansible playbook, anyway.
  • mherrmann 6y ago
    How do you debug (in dev or prod), or in production, do things such as inspecting log files, monitoring system resources, running necessary commands if there is something urgent...? In my experience, Docker just makes everything more tedious without tangible benefits. I personally lose nothing by not using Docker and am much faster in everything I do.
    • piaste 6y ago
      > How do you debug (in dev or prod) VSCode lets you debug inside a Docker container. > or in production, do things such as inspecting log files Logs should be sent out of the container, either directly (mount a /logs folder, push them to Sentry / ELK / etc.) or by simply logging to stdout and having Docker send the logs where you want. > monitoring system resources Docker processes are still processes. They show up in `top` and friends just fine. > running necessary commands if there is something urgent...? `Docker exec my_container <insert command here>`. Although I've never even considered doing that. For the past few years the resolution to a production bug has always been "rollback to the previous image, then if any data got borked fix it in the database".
• fmakunbound 6y ago
  It’s too fucking hard to setup python with all its pyenv, anaconda, poetry, pip requirements.txt and what-not, that’s the best use case for Docker here. Get it to work once and forget about it.
  • mherrmann 6y ago
    I also dislike all those tools you mention. So often, I feel like devs use a tool "because it's cool" but fail to appreciate the complexity it adds - not just for themselves but for others. The frustration expressed by your comment is a perfect example of this. In my ideal world, everybody would just use what comes with vanilla Python: python3 -m venv venv source venv/bin/activate (Or `call` on Win) pip install -Ur requirements.txt
    • procinct 6y ago
      Venv is great for keeping dependencies separate but if your local machine doesn’t have the same version of Python as your server then you’ll need pyenv.
      • RMPR 6y ago
        Not necessarily, you can use a CI pipeline to verify that your project can be build with many Python versions. This is the workflow I use here[0]. It makes catching and fixing breaking changes easier. Plus, I'm confident the core team is not likely to introduce a painful breaking change (think Python 2 -> 3) soon[1] 0: https://github.com/rmpr/atbswp https://github.com/rmpr/atbswp 1: https://mobile.twitter.com/gvanrossum/status/1306082472443084801?lang=en https://mobile.twitter.com/gvanrossum/status/130608247244308...
        • imnotjames 6y ago
          That's great for libraries but isn't it a pretty huge cost for an app you only really are going to be running on one version at a time? Dealing with BC breaks for versions you aren't intending to run in production seems like unnecessary overhead.
• pbar 6y ago
  This kind of thinking is why engineering departments have tech debt.
  • mherrmann 6y ago
    I get your point but don't think you're getting mine. In a bigger project / organization? Yes, let's have those processes and tools. But for simple apps as described in the article? Use a correspondingly simple solution. As always in software development, it's all about context.
    • pbar 6y ago
      I get your point, and I disagree on opinion. I and many others have had success using containerized devenvs on projects both large and small, and have likewise felt some pain with respect to repeatability when not - especially with the Python stack. Containers are synonymous with repeatability. Your future self is just another collaborator, and they’ll appreciate it down the line when they’ve got a new laptop and new env, for example ;)
    • kcartlidge 6y ago
      I agree. With you and with the other opinions (why should only one perspective be right?). I've been the lead developer on teams where I introduced Docker to solve consistency/reproducibility issues in AWS and Azure. I've also done smaller applications in DotNet Core, Go, Node, Python, and Ruby. In those cases I've use other alternatives, including: - Known Linux version, with git push-to-deploy (my favourite) - Packer (the server), with embedded codebase (still quite simple) - Docker (for most non-trivial deployments) - Known Linux version, with chef or ansible (as an alternative to Docker) - Terraform the machine, upload the codebase, run scripts on the server (ugh) Every method had it's place, time, and reason. If possible, for simplicity, I'd go with the first option every time and then the others in that order. The thing is, though, I may have an order of preference but that is totally overridden by the requirements of the project and whether or not the codebase is ever to be shared. For solo projects and small sites, I've not benefited from Docker as I have never had any server/OS issues (and I've been doing dev stuff for decades). However the moment there was a need for collaborators or for pulling in extra dependencies (and here is the crunch point for me) such as headless browsers or other such 'larger' packages, then I would move on to either Packer/Terraform for fairly slow-changing deployment targets or Docker for fast-changing targets, as otherwise I inevitably started to find subtle issues creeping in over time. In other words keep it simple while (a) you can and (b) you don't need to share code, but complexity inevitably changes things.
• ehou 6y ago
  For multi tenant deployments. Each tenant gets its own docker-compose in a directory on prod and voila: 100% same code base (same Docker image) and good separation between tenants
  • mherrmann 6y ago
    Yes, that's an example where Docker might make sense. But I don't think it's very common, or what OP was writing about.
  • adgud 6y ago
    Multi-tenant deployments with Docker make sense _only_ if you trust all of the tenants, since it is trivial to take control of the host if you have write access to Docker socket. You may say that it can be mitigated with some wrapper scripts with limited commands, but then you have to maintain them and we can all agree that homebrew security is very hard to do correctly.
    • ehou 6y ago
      True. We trust all tenants. It's a way of separating tenants beyond database schemas or organization_id columns.
• laingc 6y ago
  This is a pretty common sentiment on HN and I’m sure you have your reasons for it, but to me personally it just seems bonkers. Docker is an absolutely game-changing tool for me. I can’t count the number of times it’s saved me from completely screwed up system libraries, tools installing config files in weird places, conflicting versions of this or that, or other versions of system pollution. I write a simple docker-compose.yml and I’m good to go - Docker takes care of sand boxing, networking, mounts - the works. As I said before, you must have your reasons for your view, but my own experience is the polar opposite. edit: typo
  • mherrmann 6y ago
    Sure, we're all just talking about our personal experiences. I just haven't run into the problems you describe. I use virtual environments to keep my system installation clean, and PostgreSQL / Nginx are so stable that the version I happen to have on my dev machine usually works with all my projects dating back to 2014 (but kept up-to-date).
    • laingc 6y ago
      Do you by any chance often have cause to compile and use tools or libraries written by academics? Or regularly use projects that aren’t as big and well-developed as Postgres and Nginx?
      • mherrmann 6y ago
        No. But I can see how Docker would be useful in such cases.
    • growse 6y ago
      > Sure, we're all just talking about our personal experiences. I just haven't run into the problems you describe. I use virtual environments to keep my system installation clean, and PostgreSQL / Nginx are so stable that the version I happen to have on my dev machine usually works with all my projects dating back to 2014 (but kept up-to-date). One of the most important thing docker provides is an abstraction between the application and the OS. If you don't need the abstraction, then it's only going to look like cost. On the other hand, being able to write an some code in python 3.9 and deploy it anywhere across a heterogeneous production environment without fighting with the OS about how many versions of python it has installed is a useful thing.
• jskrablin 6y ago
  Using Docker will provide you a complete runtime env which will in most cases 'just work' on a variety of hosts. It's easy to reproduce and easy to distribute. You can easily setup other dependencies and connect everything together using docker compose, creating a portable 'production env'. And that's important when there's > 1 person working on a project. And it won't break the host machine because someone ran sudo pip install -r requirements.txt by mistake.
• zer01 6y ago
  For me it’s actually more about isolation and security. I touch and build a lot of random code and docker gives me some good assurances that a transient dependency of a dependency somewhere in the pits of NPM install hell won’t do things like install malware on my computer. Sandboxing/isolation is a very good mitigation against software supply chain worries IMO
• spech 6y ago
  Thank you! I would wish that there are more projects having things like ansible modules or similar. You can run it for you local dev on vagrant images and deploy it on the server directly. Less layers. Less differences between environments. No messing with docker bypassing ufw and other system configs.
• neurostimulant 6y ago
  With docker, I can move or upgrade the host server without worrying about compatibility of my apps. Before using docker, every time I migrate to a new version of os or database (which I do every 3-4 years per host), I would often run into small problems in some old apps that must be fixed, which took time because I had to dig into the app again and re-learn the context before I can patch the small issue. After moving everything to docker, it's no longer a problem. As long as the host can run docker (be it docker compose or kubernetes) my apps would run on it.
  • mrweasel 6y ago
    You still need to do those updates inside your container. I deal with a number of clients who like Docker because it gives them “a stable environment” but they don`t plan image upgrades. I work products based on the official Python Docker image, but the base image containing the version of Python they want was release in April and haven`t received updates since. Developers forget that releasing software as Docker images mean that they are now responsible for patch management.
    • neurostimulant 6y ago
      Yes, but now I can do the update in a separate schedule instead of being forced to do it right there and then when performing host upgrade or migrating the app to a different host.
• viraptor 6y ago
  > You can run everything without Docker to remove an extra level of indirection. ... and add questions like: are all paths correct for this machine, are binary lib dependencies the same as in production, are the OS versions compatible between dev and prod, is my local runtime version compiled with same options? The great example where the indirection adds value is: every developer has the same environment and the CI is the same, regardless of personal preferences in systems.
  • mherrmann 6y ago
    If you have to answer such questions then yes, Docker is probably a good solution for them. I find that in my pretty vanilla web development, I am not faced with those questions. Most of my stuff is just Python, Django, PostgreSQL, Nginx and a few not very exciting Python dependencies. (This is in several independent projects, working as a sole developer, since 2014 and let's say ~100.000 users of my SaaS apps.)
    • viraptor 6y ago
      In my experience you'll just run into them one day by accident. My last one was actually about differences in postgres drivers talking to sqlalchemy between Debian and Fedora. I agree it's likely a "pick your poison" situation, but these days I prefer defaulting to containers (and investing time upfront) for just about everything to prevent those problems rather than debug them once they happen.
    • mockingbirdy 6y ago
      What do you use for the frontend part? I'm using Vue or React + django-drf, but it always feels like it could be much simpler for most SaaS applications. Using Django forms feels too limited, on the other hand.
    • wp381640 6y ago
      I could tell you were a solo developer before getting to this comment. A lot of the problems docker solves is as soon as you need to fit out a team with a mix of environments and split ops from dev I still use it for individual projects because I never plan on maintaining everything forever - makes it easier to grow to a 1+1 team or hand the project to someone else or solicit contributions Also builds your experience for when you do work in teams
• zendist 6y ago
  I had great use of Docker even in my personal RPI projects, as Python modules can be difficult to get compiled and in a usable state on raw Raspbian. In projects at work I personally only recommend Docker when we need Hyper-V isolation or K8s is hosting the product (these are .NET projects, which work really well when published as self-contained binaries).
• willemmerson 6y ago
  At work I had to set up a server with django, postgres, SSL, rabbitmq, celery and a few other things. I did it in both ansible and docker-compose, the ansible version was 1500 lines and docker-compose version was 100 lines. A lot of the ansible stuff is dealing with users, groups, services, supervisord, virtualenvs and repositories - none of this is required in docker as containers restart themselves, users are already set up, you don't need virtualenv, and using containers avoids the need to use repositories to get specific software versions. Also the docker-compose version was much faster to pick up changes, vs minutes for ansible. I don't find docker that useful for developing on a day to dat basis however, it just gets in the way and generally you're not going to need a webserver, SSL, celery, rabbitmq in development anyway. It is useful to debug things locally though. The other reason to use docker is state: while using ansible we got bitten by old dependencies hanging around because ansible doesn't exactly mirror your config, it can really only add stuff. Whereas docker actually wipes the slate clean every single deploy so your server is an exact mirror of your docker-compose file. This is really important.
  • hibbelig 6y ago
    You can tell Ansible that a package should be removed. You just have to remember to do it. If there was ever a statement in your playbook that a package should be added then you can’t assume it will disappear automatically - you have to tell Ansible to remove it.
  • knowhy 6y ago
    Most of the stuff you would be doing with Ansible happens inside the Dockerfiles. So comparing lines of code of your Ansible codebase with just the docker-compose file is not fair towards Ansible.
    • willemmerson 6y ago
      That 100 lines includes the Django docker file which is only 5 lines long, there are no other docker files, at least none I have to worry about.
  • paulryanrogers 6y ago
• antidocker 6y ago
  I feel OP would be better off with Ansible.
• strzibny 6y ago
  Yep. I agree. Docker has its place, but nothing wrong with keeping everything one layer short. It's how I run my side-projects. Just a small Bash setup.sh script and you are good to go. Docker solves the annoyance of system dependencies, but you still have to deal with it in your Docker build. As I always say, know what you are doing + know to debug everything. If somebody is more comfortable with Docker, I think no problem there either :). Sometimes it solves a real pain, sometimes don't (having an Elixir release or Java jar makes me not to really use containers). If you are on Docker, I recommend looking into Podman (maybe even together with runc -> today trending on HN). Don't run your containers under root and run your app inside the container under a specific user as well. Shameless plug: I am now writing a book on application deployment and will show both approaches, Docker and Docker-less[0]. [0] https://deploymentfromscratch.com/ https://deploymentfromscratch.com/
• matsemann 6y ago
  Disagree. Not using docker I have to learn about the specifics of how the hosting company runs my software. With docker it's 30 lines I can copy from an earlier project and I have full control.
• ldng 6y ago
  Well, maybe you are careful and meticulous and are the only person to deploy. But when you are working in a team where anybody can deploy, maybe not everybody as meticulous as you. Using Docker makes it easy for anyone to rollback anything (yeah, yeah, you know those kind of issues that should not happen but it still do ...), and that's a game changer.
• nickjj 6y ago
  > usage of Docker made it more expensive to host It depends on how you're using it. Docker itself isn't going to increase your hosting costs. If you rent a server for $20 a month, it's still $20 a month with or without Docker being installed.
  • adgud 6y ago
    What you usually can't avoid though is that you have to account for much bigger disk usage for the image storage - both on local and target machine.
• Youden 6y ago
  I'm not trying to pick on you personally here but this comment, like many I see on HN, seems overly adversarial and could have made the same arguments in a less combatative way. There's a lot of emphasis on your own perspective: "*I* don't get", "*I've* been doing that", "Not once have *I* missed Docker", "making *me* less productive", "*my* personal experience*" And an assumption that it applies to others: "Just use the same [...]", "I feel OP would be better off [...]". The comment could provoke a more productive, thoughtful discussion by focusing on understanding rather than on pushing particular views and practices. For example: - "I don't get why people use Docker" -> "Could anybody share their reasoning for using Docker?" - "You can run everything without Docker to remove an extra level of indirection" -> "What benefits do you see in exchange for this extra level of indirection?" - "Not once have I missed Docker." -> "I haven't found much use for Docker and would like to understand those who have" - "I have taken over [...] making me less productive" -> "I've only encountered downsides in my own experience with Docker, could anybody share what the upsides may have been for others involved?" - "Again in my personal experience, this simply isn't an issue" -> "I'd be interested to hear more about the benefits as I haven't encountered issues before" - "In short, I feel OP would be better off without Docker" -> "I wonder if the OP really needs Docker, perhaps they could be more productive without it" If we all wrote like this (and I'm certainly not perfect myself), HN could be a much friendlier, more welcoming place.
  • christiansakai 6y ago
    I think I’m fine with the commenter’s style of writing.
  • mherrmann 6y ago
    I understand your concern and agree that I am being extremely direct. It's not out of spite. On the one hand, it comes from my frustration with what I as an engineer perceive to be wrong advice. It feels to me like OP is using Docker because he read a blog post about it somewhere, never seriously thought about alternatives, then stuck with it and now recommends it to others, repeating a cycle where software gets worse and worse. And on the other hand, being direct is more concise. We all have little time and I feel that to get a point across on the internet, one has to get it across quickly and succinctly. I have no problem with my views being criticised and I'll be happy to concede where I was wrong if someone makes a clear and harsh point. But either way, I'm sorry if I offended you in particular or others who did not comment; It was not my intention. Having read this thread, I think much of the difference of opinion comes from different contexts. People who like Docker often seem to be working in larger teams, which is totally fine and they're probably right to use Docker. I just couldn't help but disagree with a statement that I felt pertains to an area I work in, which is small SaaS of tiny teams with usually no scalability requirements.
• krzepah 6y ago
  "Just use the same Python version and a roughly similar PostgreSQL version in development and production, and you're good to go" -> And that's more time than it would take to re-use that Dockerfile and docker-compose that i literally wrote once and go to once in a while.
• JeremyNT 6y ago
  Docker makes deployments easy. In almost all situations I've encountered, it's easier to 'git pull foo && cd foo && docker-compose up' than it is to install and configure a mishmash of individual services to work in tandem. A Dockerfile may be complex for any piece of the stack. But now this complexity is not part of deploying the application. Each component has its own build process which is decoupled from the others. And even a really complex installation process results in a deployment process that works the same way. If you can't see these benefits then maybe you haven't had to deal with complex software installation. Or maybe you haven't had to run the same stack on your laptop that you run in production. But if you ever get to the point that you want to deploy something in more than one place, or more than once to a single place, it really makes things much easier.
• michaelcampbell 6y ago
  Docker isn't necessarily for scale; it's for packaging.
• KronisLV 6y ago
  In my personal experience, there are a few reasons for using Docker: 1) By installing packages on the system directly, you can't be sure that a future update won't break it. For example, when using Tomcat or a similar application server, new updates sometimes deprecate or disable functionality in the older versions, or add new configuration parameters, that the configuration from the older versions won't have, thus leading to weird behaviour. This will be especially noticeable, if you set up development, test or production environments some time apart, or in different locations, where the available packages may not be 100% consistent (i've had situations where package mirrors are out of date for a while). 2) Furthermore, if you maintain the software long term, it's likely that your environments will have extremely long lists of configuration changes, managing which will be pretty difficult. This results in the risk of either losing some of this configuration in new environments, or even losing some of the knowledge over time, if your approach to change management isn't entirely automated (e.g. Ansible with the config in a Git repo and read-only access to the servers), or you don't explain why each and every change is done. 3) Also, it's likely that if you install system packages (e.g. Tomcat from standard repositories instead of unzipping it after downloading it manually), it'll be pretty difficult for you to tell where the system software ends and where the stuff needed by your app begins. This will probably make migrating to newer releases of the OS harder, as well as will complicate making backups or even moving environments over to other servers. 4) If your application needs to scale horizontally, that means that you'll need multiple parallel instances of it, which will once again necessitate all of the configuration that your application needs to be present and equal for all of them. You can of course do this with Ansible, but if you don't invest the time necessary for it, then it's likely that inconsistencies will crop up. In the case of Knight Capital, this caused them to lose more than 400 million dollars in less than an hour: https://dealbook.nytimes.com/2012/08/02/knight-capital-says-trading-mishap-cost-it-440-million/ https://dealbook.nytimes.com/2012/08/02/knight-capital-says-... Edit: It was stated in this case that "scale is not an issue" and therefore this point could be ignored. But usually scale isn't an issue, until it suddenly is. 5) Also, you'll probably find that it'll be somewhat difficult to have multiple similar applications deployed on the server at the same time, should they have any inconsistencies amidst them, such as needing a specific port, or needing a specific runtime on the system. For example, if you've tested application FOO against Python 3.9.1, then you'll probably need to run it against it in production, whereas sif you have application BAR that's only tested with Python 3.1.5 and hasn't been updated due to a variety of complex socioeconomical factors, then you'll probably need to run it against said older runtime. 6) Then there's the question of installing dependencies which are not in the OSes package repositories, but rather are available only in the npm (for Node.js), pip (for Python) or elsewhere, where you're also dealing with different mechanisms for ensuring consistency and making sure that you're running with exactly the version that you have tested the application on can be a bit of a pain. For an example of this going really wrong, see the left-pad incident: https://www.theregister.com/2016/03/23/npm_left_pad_chaos/ https://www.theregister.com/2016/03/23/npm_left_pad_chaos/ Essentially, it's definitely possible to live without containers and still have your environments be mostly consistent (for example, by using Ansible), but in my experience it's just generally harder than to tell an orchestrator (preferably Docker Swarm/Hashicorp Nomad, because Kubernetes is a can of worms for simple deployments) that you'd like to run application FOO on servers A, B and C with a specific piece of configuration, resource limits, storage options and some exposed ports. I feel like this is a largely issue of tooling and our development approaches, NixOS attempts to solve this, but i can't comment on how successful it is: https://nixos.org/ https://nixos.org/
• tyingq 6y ago
  It's a crude hammer, but being able to ensure everything is mostly the same in dev/test/prod, even in the middle of host upgrades...is nice.
• qudat 6y ago
  Docker compose and docker machine make deployments so simple. For development I don’t want all my side projects to share the same database, similar to them using the same python install.
• mixmastamyk 6y ago
  It's not about docker per-se but repeatable, immutable infrastructure, a great boon to productivity and portability to local or cloud environment. "Cattle vs. pets" Sure, you can do similar with ansible or puppet, but use something. SSHing into boxes to fix things wasn't good enough in the 90's, one reason I looked into building .debs back before the turn of the century. cough
  • antidocker 6y ago
    "repeatable" BS!!!
• girvo 6y ago
  I can run disparate environments across multiple language runtimes without having to fight version issues or wrangle multiple VMs. That’s why, for me at least.
• 1337shadow 6y ago
  If you're not using docker then you're building in production instead of in CI. Also, that means you're maintaining a deployment instead of building a fresh one every time you deploy. This means leftover files will pile up and that is certainly not tested upstream at any time: tests run on fresh ephemeral deployments in general.
• athosblade 6y ago
  I totally agree with you in regards that Docker is overkill if your app is a Django app that connects to database. I can't image a world where running python manage.py runserver is less productive than spinning up two containers. But once you need stuff like Celery workers, message brokers, and multiple databases for sharding, Docker is a life-saver.