5 ms·

There's a lot of "tool" selections in that repo. If anyone is looking for ready to go web app examples aimed at both development and production with Docker Com

by nickjj 4y ago

There's a lot of "tool" selections in that repo. If anyone is looking for ready to go web app examples aimed at both development and production with Docker Compose, I maintain: - https://github.com/nickjj/docker-flask-example - https://github.com/nickjj/docker-rails-example - https://github.com/nickjj/docker-django-example - https://github.com/nickjj/docker-node-example - https://github.com/nickjj/docker-phoenix-example About once a week or so I update them to their latest versions for everything. The examples use a combination of services for each tech stack such as web + worker + postgres + redis + esbuild + tailwind. The Rails example is set up for Hotwire and runs Action Cable as a dedicated service along with Sidekiq where as the Flask and Django examples use Celery as a worker. You can easily swap things out since the examples are starter projects that you can clone + rename (they all come with a rename script), you're meant to customize them to build your app on top of.

decide1000 4y ago
Thanks! I will use this.
fbdab103 4y ago
This looks great. Definitely a few idioms I will have to explore further. I can use Docker in a basic sense, but it is amazing to me how much black arts still exists for what has become a cornerstone of modern deployment. Lots of conflicting/dated advice about best practices. Unsure which advice is still required/applies to podman, etc.
- sneak 4y ago
  > but it is amazing to me how much black arts still exists for what has become a cornerstone of modern deployment. What?! Unless you are using complicated networking, docker is simple. Dockerfiles are essentially just annoying-syntax shell scripts. We have plenty of black arts in computing. Docker isn't one of them.
  - fbdab103 4y ago
    So, am I still supposed to specify UID and GID? Should I be using Alpine or Debian? How do I handle loading certificates? Should I use an override config for development? In this thread, someone was indicating that the naive volume option does not work over SSH. Also in this thread was a request to pin by tag and not hash. Do I still need to worry about Docker blowing a hole in iptables? Maybe not black magic, but there are a lot of subtle optimizations for which there is much conflicting guidance.
    - smaccona 4y ago
      This is how people end up using Kubernetes, despite its warts. [edit: added second clause]
      - doix 4y ago
        Kubernetes doesn't answer any of the parents questions though, does it? You still need a Dockerfile for k8s and all those questions still apply.
    - nickjj 4y ago
      > So, am I still supposed to specify UID and GID? If you run your container as a non-root user and create a user without setting a UID / GID it'll default to 1000:1000, so unless your Docker host's user isn't 1000:1000 then things work out of the box. A lot of this is general Linux knowledge around user / file permissions, not so much Docker. > Should I be using Alpine or Debian? Debian, no contest in my opinion. > How do I handle loading certificates? You can volume mount them or deal with SSL certificates in a way where Docker isn't involved such as running nginx on your Docker host directly or putting a load balancer in front of your app and handling SSL there. > Should I use an override config for development? You can use the same docker-compose.yml file in all environments and tweak things with environment variables. The compose file supports variable interpolation. Docker Compose profiles also let you control which services to run in each environment, it's even configurable by a single env variable. > Do I still need to worry about Docker blowing a hole in iptables? If you use -p 8000:8000, yes this will publish the port in a way where the outside world can access it. Likewise without Docker if you edit iptables to allow that port it will too. I wouldn't classify this as blowing a hole in iptables. This is "user configured application to make a port open to the world". > Maybe not black magic, but there are a lot of subtle optimizations for which there is much conflicting guidance. In the end my example apps address most of these issues for you. You're on your own with certificates since that varies on your deployment, but everything else is fully set up and ready to go and it protects yourself from blowing a hole in iptables since it only publishes the port to localhost by default, not 0.0.0.0. This would let nginx or another web server access it directly on your Docker host but no one else.
davepeck 4y ago
These are excellent, thank you. I maintain similar Django and Flask + compose stacks on behalf of the startup studio I work for so it’s fun to compare notes. For our Django stack, for instance, we have also settled on Postgres, on celery+redis, and on whitenoise. black/flake8/isort also seem universally agreeable. We also throw in pyright and generally make extensive use of type hints. For the front-end, we’re currently Create React App with TypeScript and Tailwind. I’d love for us to move away from CRA, so your use of esbuild is helpful to see. (I’d personally be happy using HTMX or Turbo/Stimulus but for the moment a JSON API backend with a React SPA front-end seems more comfortable for more of the CTOs who hop on board.) We also supply some minimally opinionated glue at the API layer. On the back end we have a base View that provides a few helper methods for transmuting invalid Form instances to nice JSON replies that the TypeScript API invocation code works with gracefully. (We used to push DRF but have lots of feedback from older startups than ran with it and had regrets down the road.)
- holoduke 4y ago
  no offense, but man, the amount of tooling you guys are using sounds to me insane. How is a person able to oversee and understand everything. The older i get the more i feel distantiated and disconnected to these modern practices. I am afraid that if I ever have to find a new workspace i wont be able to succeed because of this.
  - zmmmmm 4y ago
    I definitely relate to this but frequently challenge myself and then overcome it. You just have to bite the bullet and carve out some time to try them out. The key thing to understand is that all these things are shrouded in incomprehensible jargon and alien sounding names that make it incredibly intimidating to get started. But the fact is, it is a wide but shallow pool of jargon sitting on top of the same old computing fundamentals that have been around since the 1970s. You will find if you know your fundamentals, then the jargon is far less difficult to overcome than it seems. You just need a bit of exposure and if you spend any amount of time playing with the tech it just starts to happen by osmosis. (If there are fundamentals you aren't solid on, treat it as an opportunity to bed that in - even these things are not generally super complex in the end).
afrohorse 4y ago
Any golang?
- nickjj 4y ago
  Nope, I haven't run any Go code in production that was running in Docker. Go's ecosystem is also pretty fragmented so it would be difficult to create a solution that lots of folk would be happy with. I get the impression most folks who use Go want to pick everything themselves (not a bad thing, just something I noticed). Plus I'm only 1 person doing this in my free time with no income sources attached to these projects, I don't have the capacity. With that said, most of the apps are very similar when it comes to the Docker bits. You could take any of the examples and replace XXX with Go. I'd suggest basing it off the Phoenix example because that one covers using a 2nd Docker build stage to create a release. With Go being able to compile a self-contained binary that pattern of using a build stage to copy that over would be similar.
hobs 4y ago
Taking a quick look and nothing I saw manages ssl - do you even package that or do you have that as entirely different steps?
- Ambolia 4y ago
  Not the same guy, but many cloud providers take care of ssl in their load balancers.
- nickjj 4y ago
  It's not included with these projects, as someone else mentioned in a reply to you, it can be handled in a number of different spots. For example, a load balancer running 1 level above your server or perhaps nginx running directly on your Docker host without Docker. A while back I wrote up why I prefer running nginx outside of Docker at: https://nickjanetakis.com/blog/why-i-prefer-running-nginx-on-my-docker-host-instead-of-in-a-container https://nickjanetakis.com/blog/why-i-prefer-running-nginx-on...
  - hobs 4y ago
    Your paragraph on SSL is basically what I have come to understand so thank you, I just thought I must be stupid or something.
quest88 4y ago
Does this start rails app and postgre db on the same server?
- nickjj 4y ago
  By default yes but you can change a single environment variable to not run Postgres or Redis through Docker Compose. You can also very easily choose to run Puma and Sidekiq on different servers. It leverages Docker Compose v2 profiles. I'd suggest watching and reading: https://nickjanetakis.com/blog/a-guide-for-running-rails-in-docker https://nickjanetakis.com/blog/a-guide-for-running-rails-in-... It's a full end to end walk through of the example Rails app and how it all works when it comes to Docker and Docker Compose. I just recorded it a month ago, it covers everything with up to date info vs the code in the repo.
sho 4y ago
Awesome repos, starred. Do you have any preferences for next steps, such as deploy/management? And if you’ve seen it, what do you you think of https://github.com/mrsked/mrsk https://github.com/mrsked/mrsk?
- nickjj 4y ago
  Thanks. Yep I've seen mrsk and even briefly chatted with DHH about it a day or 2 after he open sourced it. I have mixed feelings: On one hand I think it's fantastic Rails is starting to take on deployment officially. This is going to lead things to a better place in the long run. Generally speaking DHH has a really good track record for making things that feel good to use. On the other hand, I think the project is trying to reinvent too many things that already exist and doesn't account for Docker Compose. For example, for literally the last 7 years I've been deploying any Dockerized web application with Docker Compose to 1 or more servers with about 15-20 lines of YAML using Ansible to set up the server and git to deploy the code. It doesn't matter if it's Rails, Flask, Django, Phoenix, Node, Go or whatever. It's all the same. Those 20 lines also include everything from taking a blank slate Debian / Ubuntu box to production ready, complete with self managed system updates, locking down SSH, iptables, various server configurations / optimizations, nginx, HTTPS, database backups, sane logging and everything else you'd expect. I don't have anything ready yet but I've been slowly working on https://nickjanetakis.com/courses/deploy-to-production https://nickjanetakis.com/courses/deploy-to-production to assemble all of this into a course.

by nickjj 4y ago

decide1000 4y ago

Thanks! I will use this.

fbdab103 4y ago

This looks great. Definitely a few idioms I will have to explore further. I can use Docker in a basic sense, but it is amazing to me how much black arts still exists for what has become a cornerstone of modern deployment. Lots of conflicting/dated advice about best practices. Unsure which advice is still required/applies to podman, etc.

sneak 4y ago
> but it is amazing to me how much black arts still exists for what has become a cornerstone of modern deployment. What?! Unless you are using complicated networking, docker is simple. Dockerfiles are essentially just annoying-syntax shell scripts. We have plenty of black arts in computing. Docker isn't one of them.
- fbdab103 4y ago
  So, am I still supposed to specify UID and GID? Should I be using Alpine or Debian? How do I handle loading certificates? Should I use an override config for development? In this thread, someone was indicating that the naive volume option does not work over SSH. Also in this thread was a request to pin by tag and not hash. Do I still need to worry about Docker blowing a hole in iptables? Maybe not black magic, but there are a lot of subtle optimizations for which there is much conflicting guidance.
  - smaccona 4y ago
    This is how people end up using Kubernetes, despite its warts. [edit: added second clause]
    - doix 4y ago
      Kubernetes doesn't answer any of the parents questions though, does it? You still need a Dockerfile for k8s and all those questions still apply.
  - nickjj 4y ago
    > So, am I still supposed to specify UID and GID? If you run your container as a non-root user and create a user without setting a UID / GID it'll default to 1000:1000, so unless your Docker host's user isn't 1000:1000 then things work out of the box. A lot of this is general Linux knowledge around user / file permissions, not so much Docker. > Should I be using Alpine or Debian? Debian, no contest in my opinion. > How do I handle loading certificates? You can volume mount them or deal with SSL certificates in a way where Docker isn't involved such as running nginx on your Docker host directly or putting a load balancer in front of your app and handling SSL there. > Should I use an override config for development? You can use the same docker-compose.yml file in all environments and tweak things with environment variables. The compose file supports variable interpolation. Docker Compose profiles also let you control which services to run in each environment, it's even configurable by a single env variable. > Do I still need to worry about Docker blowing a hole in iptables? If you use -p 8000:8000, yes this will publish the port in a way where the outside world can access it. Likewise without Docker if you edit iptables to allow that port it will too. I wouldn't classify this as blowing a hole in iptables. This is "user configured application to make a port open to the world". > Maybe not black magic, but there are a lot of subtle optimizations for which there is much conflicting guidance. In the end my example apps address most of these issues for you. You're on your own with certificates since that varies on your deployment, but everything else is fully set up and ready to go and it protects yourself from blowing a hole in iptables since it only publishes the port to localhost by default, not 0.0.0.0. This would let nginx or another web server access it directly on your Docker host but no one else.

davepeck 4y ago

These are excellent, thank you. I maintain similar Django and Flask + compose stacks on behalf of the startup studio I work for so it’s fun to compare notes. For our Django stack, for instance, we have also settled on Postgres, on celery+redis, and on whitenoise. black/flake8/isort also seem universally agreeable. We also throw in pyright and generally make extensive use of type hints. For the front-end, we’re currently Create React App with TypeScript and Tailwind. I’d love for us to move away from CRA, so your use of esbuild is helpful to see. (I’d personally be happy using HTMX or Turbo/Stimulus but for the moment a JSON API backend with a React SPA front-end seems more comfortable for more of the CTOs who hop on board.) We also supply some minimally opinionated glue at the API layer. On the back end we have a base View that provides a few helper methods for transmuting invalid Form instances to nice JSON replies that the TypeScript API invocation code works with gracefully. (We used to push DRF but have lots of feedback from older startups than ran with it and had regrets down the road.)

holoduke 4y ago
no offense, but man, the amount of tooling you guys are using sounds to me insane. How is a person able to oversee and understand everything. The older i get the more i feel distantiated and disconnected to these modern practices. I am afraid that if I ever have to find a new workspace i wont be able to succeed because of this.
- zmmmmm 4y ago
  I definitely relate to this but frequently challenge myself and then overcome it. You just have to bite the bullet and carve out some time to try them out. The key thing to understand is that all these things are shrouded in incomprehensible jargon and alien sounding names that make it incredibly intimidating to get started. But the fact is, it is a wide but shallow pool of jargon sitting on top of the same old computing fundamentals that have been around since the 1970s. You will find if you know your fundamentals, then the jargon is far less difficult to overcome than it seems. You just need a bit of exposure and if you spend any amount of time playing with the tech it just starts to happen by osmosis. (If there are fundamentals you aren't solid on, treat it as an opportunity to bed that in - even these things are not generally super complex in the end).

afrohorse 4y ago

Any golang?

nickjj 4y ago
Nope, I haven't run any Go code in production that was running in Docker. Go's ecosystem is also pretty fragmented so it would be difficult to create a solution that lots of folk would be happy with. I get the impression most folks who use Go want to pick everything themselves (not a bad thing, just something I noticed). Plus I'm only 1 person doing this in my free time with no income sources attached to these projects, I don't have the capacity. With that said, most of the apps are very similar when it comes to the Docker bits. You could take any of the examples and replace XXX with Go. I'd suggest basing it off the Phoenix example because that one covers using a 2nd Docker build stage to create a release. With Go being able to compile a self-contained binary that pattern of using a build stage to copy that over would be similar.

hobs 4y ago

Taking a quick look and nothing I saw manages ssl - do you even package that or do you have that as entirely different steps?

Ambolia 4y ago
Not the same guy, but many cloud providers take care of ssl in their load balancers.
nickjj 4y ago
It's not included with these projects, as someone else mentioned in a reply to you, it can be handled in a number of different spots. For example, a load balancer running 1 level above your server or perhaps nginx running directly on your Docker host without Docker. A while back I wrote up why I prefer running nginx outside of Docker at: https://nickjanetakis.com/blog/why-i-prefer-running-nginx-on-my-docker-host-instead-of-in-a-container https://nickjanetakis.com/blog/why-i-prefer-running-nginx-on...
- hobs 4y ago
  Your paragraph on SSL is basically what I have come to understand so thank you, I just thought I must be stupid or something.

quest88 4y ago

Does this start rails app and postgre db on the same server?

nickjj 4y ago
By default yes but you can change a single environment variable to not run Postgres or Redis through Docker Compose. You can also very easily choose to run Puma and Sidekiq on different servers. It leverages Docker Compose v2 profiles. I'd suggest watching and reading: https://nickjanetakis.com/blog/a-guide-for-running-rails-in-docker https://nickjanetakis.com/blog/a-guide-for-running-rails-in-... It's a full end to end walk through of the example Rails app and how it all works when it comes to Docker and Docker Compose. I just recorded it a month ago, it covers everything with up to date info vs the code in the repo.

sho 4y ago

Awesome repos, starred. Do you have any preferences for next steps, such as deploy/management? And if you’ve seen it, what do you you think of https://github.com/mrsked/mrsk https://github.com/mrsked/mrsk?

nickjj 4y ago
Thanks. Yep I've seen mrsk and even briefly chatted with DHH about it a day or 2 after he open sourced it. I have mixed feelings: On one hand I think it's fantastic Rails is starting to take on deployment officially. This is going to lead things to a better place in the long run. Generally speaking DHH has a really good track record for making things that feel good to use. On the other hand, I think the project is trying to reinvent too many things that already exist and doesn't account for Docker Compose. For example, for literally the last 7 years I've been deploying any Dockerized web application with Docker Compose to 1 or more servers with about 15-20 lines of YAML using Ansible to set up the server and git to deploy the code. It doesn't matter if it's Rails, Flask, Django, Phoenix, Node, Go or whatever. It's all the same. Those 20 lines also include everything from taking a blank slate Debian / Ubuntu box to production ready, complete with self managed system updates, locking down SSH, iptables, various server configurations / optimizations, nginx, HTTPS, database backups, sane logging and everything else you'd expect. I don't have anything ready yet but I've been slowly working on https://nickjanetakis.com/courses/deploy-to-production https://nickjanetakis.com/courses/deploy-to-production to assemble all of this into a course.