5 ms·

> How does Docker handle asynchronous intermittent processes? Not good, actually. It is well known in Docker 101 that thou shall not run cron in the same contai

by cle 5y ago

> How does Docker handle asynchronous intermittent processes? Not good, actually. It is well known in Docker 101 that thou shall not run cron in the same container as the main process. You can run cron just fine in Docker containers. Whether or not that's a good idea depends on what the cron is doing, what your application is doing, your runtime requirements, etc. Running cron in your container is definitely way simpler than the other options involving container scheduling.

codeflo 5y ago
This. The advice to move cron jobs outside has its place, of course. But unless you’re running at massive scale, Docker becomes easier once you stop worrying about what is and isn’t “proper container architecture”, and just remember that it’s Linux.
- roenxi 5y ago
  A voice in support - it is more important to solve the problems that you do have than the problems that the Docker community thinks you might have. But it is important to at least show a little humility and recognising what problems they are solving even.
- marcosdumay 5y ago
  > just remember that it’s Linux Hum... It's some Linux that appears out of nowhere and may go away at any time. This is the reason for most those container best practices. Anyway, this really doesn't translate to "don't", but be sure to keep the transiency in mind.
folex 5y ago
I think "1 container = 1 process" is a common misconception. There should be a separation of concerns, but there's no reason to go to extremes where it doesn't make sense. S6-overlay people lay it out pretty neat https://github.com/just-containers/s6-overlay#the-docker-way https://github.com/just-containers/s6-overlay#the-docker-way
- a_conservative 5y ago
  When docker first arrived, people were confused about how to use it. I remember seeing lots of people putting an ssh daemon in their containers!?! I like the s6-overlay approach, it's pragmatic. I haven't been using containers as much lately, but I wonder if s6-overlay's approach could be used to justify including a database into the container with an application. Is that a good idea?
  - Quekid5 5y ago
    > I haven't been using containers as much lately, but I wonder if s6-overlay's approach could be used to justify including a database into the container with an application. Is that a good idea? The answer is as always: it depends. My rule of thumb here would be: would it ever make sense to configure the application to use a different (or even just remote) database? If so, then the database should have a separate container (when using a local db). This applies most of the time. Similarly, if the database itself is an integral part of the application which it doesn't make sense to swap out or have remote... then by all means just include it and tell the user to mount their data volume to /data or whatever. Example: A single-system file indexer.
    - Groxx 5y ago
      >would it ever make sense to configure the application to use a different (or even just remote) database? If so, then the database should have a separate container (when using a local db). This can probably be further simplified to "does it run over ports? then probably yes". E.g. you'd separate your application and database, but not your database and its filesystem.
      - Quekid5 5y ago
        I'm not sure I fully understand, but with my assumptions in hand: That's bit of an implementation detail, I think. If my app is the most amazing file system indexer using PostgreSQL behind the scenes, I don't think the "port" distinction is relevant... oh, wait. You're thinking of ports as in INET vs. plain sockets? That took me a long time to get. A very technical way to put it, but thank you.
nonameiguess 5y ago
The reason for 1 container, 1 process depends upon architecting your application in such a way that it works with it. Ideally, each application thread is stateless and independent of all others, most or all of them don't require graceful shutdown, and you can encapsulate state elsewhere in persistent volumes. If you're just using a single-node container runtime engine, it's not a huge benefit, but it does make fearless restarts a lot simpler when all you have to do is stop and start the container. Think of Erlang style crash-only programming where crashing is the only way to stop a process. When you get into multi-node orchestrators, though, then the benefit becomes much more apparent, especially at a scale where nodes are guaranteed to fail every now and again. Rescheduling to another node is then, again, as simple as launch a new container from the same image on a different node. That happens a lot faster if each container only needs to start one process. It also enables you to practice chaos engineering, which is the practice of intentionally crashing your own containers every few hours. This is a good practice for security, because if an attacker ever gains a foothold in one of your containers, they'll quickly lose it, but also to just force your infrastructure people and application architects to design in such a way that you're guaranteed to be treating servers like cattle instead of pets, because you're constantly killing them and have no choice if you want your application to still work. This ensures what you're deploying is robust and repeatable and doesn't secretly depend on some specific server state an admin achieved at 1 in the morning some random Sunday hopped up on Red Bull that he'll never remember and never be able to recreate.
- cle 5y ago
  There's nothing intrinsic about crons in the same container that prevent you from rescheduling containers on other nodes, fearlessly restarting, or whacking random containers. What they're doing, how they're handling state, and their lifecycles might. Study your requirements. Whether you run your crons in the same container or a separate container depends on your system's behavior and failure/recovery scenarios. There are complexity tradeoffs--moving the cron scheduling outside of the container suddenly means you have moving parts outside of your container to test, instrument, monitor, and maintain. If you're already doing that, it's not too big of a deal, but if you aren't then I would think twice before adding all this complexity to your architecture, and make sure it's really worth it. (I can't tell from the article which approach works for their specific use cases...I'm pushing back against the general advice to "avoid cron in containers". I would default to using cron, and only move those processes to separate containers if you have a concrete reason to.)

by cle 5y ago

codeflo 5y ago

This. The advice to move cron jobs outside has its place, of course. But unless you’re running at massive scale, Docker becomes easier once you stop worrying about what is and isn’t “proper container architecture”, and just remember that it’s Linux.

roenxi 5y ago
A voice in support - it is more important to solve the problems that you do have than the problems that the Docker community thinks you might have. But it is important to at least show a little humility and recognising what problems they are solving even.
marcosdumay 5y ago
> just remember that it’s Linux Hum... It's some Linux that appears out of nowhere and may go away at any time. This is the reason for most those container best practices. Anyway, this really doesn't translate to "don't", but be sure to keep the transiency in mind.

folex 5y ago

I think "1 container = 1 process" is a common misconception. There should be a separation of concerns, but there's no reason to go to extremes where it doesn't make sense. S6-overlay people lay it out pretty neat https://github.com/just-containers/s6-overlay#the-docker-way https://github.com/just-containers/s6-overlay#the-docker-way

a_conservative 5y ago
When docker first arrived, people were confused about how to use it. I remember seeing lots of people putting an ssh daemon in their containers!?! I like the s6-overlay approach, it's pragmatic. I haven't been using containers as much lately, but I wonder if s6-overlay's approach could be used to justify including a database into the container with an application. Is that a good idea?
- Quekid5 5y ago
  > I haven't been using containers as much lately, but I wonder if s6-overlay's approach could be used to justify including a database into the container with an application. Is that a good idea? The answer is as always: it depends. My rule of thumb here would be: would it ever make sense to configure the application to use a different (or even just remote) database? If so, then the database should have a separate container (when using a local db). This applies most of the time. Similarly, if the database itself is an integral part of the application which it doesn't make sense to swap out or have remote... then by all means just include it and tell the user to mount their data volume to /data or whatever. Example: A single-system file indexer.
  - Groxx 5y ago
    >would it ever make sense to configure the application to use a different (or even just remote) database? If so, then the database should have a separate container (when using a local db). This can probably be further simplified to "does it run over ports? then probably yes". E.g. you'd separate your application and database, but not your database and its filesystem.
    - Quekid5 5y ago
      I'm not sure I fully understand, but with my assumptions in hand: That's bit of an implementation detail, I think. If my app is the most amazing file system indexer using PostgreSQL behind the scenes, I don't think the "port" distinction is relevant... oh, wait. You're thinking of ports as in INET vs. plain sockets? That took me a long time to get. A very technical way to put it, but thank you.

nonameiguess 5y ago

The reason for 1 container, 1 process depends upon architecting your application in such a way that it works with it. Ideally, each application thread is stateless and independent of all others, most or all of them don't require graceful shutdown, and you can encapsulate state elsewhere in persistent volumes. If you're just using a single-node container runtime engine, it's not a huge benefit, but it does make fearless restarts a lot simpler when all you have to do is stop and start the container. Think of Erlang style crash-only programming where crashing is the only way to stop a process. When you get into multi-node orchestrators, though, then the benefit becomes much more apparent, especially at a scale where nodes are guaranteed to fail every now and again. Rescheduling to another node is then, again, as simple as launch a new container from the same image on a different node. That happens a lot faster if each container only needs to start one process. It also enables you to practice chaos engineering, which is the practice of intentionally crashing your own containers every few hours. This is a good practice for security, because if an attacker ever gains a foothold in one of your containers, they'll quickly lose it, but also to just force your infrastructure people and application architects to design in such a way that you're guaranteed to be treating servers like cattle instead of pets, because you're constantly killing them and have no choice if you want your application to still work. This ensures what you're deploying is robust and repeatable and doesn't secretly depend on some specific server state an admin achieved at 1 in the morning some random Sunday hopped up on Red Bull that he'll never remember and never be able to recreate.

cle 5y ago
There's nothing intrinsic about crons in the same container that prevent you from rescheduling containers on other nodes, fearlessly restarting, or whacking random containers. What they're doing, how they're handling state, and their lifecycles might. Study your requirements. Whether you run your crons in the same container or a separate container depends on your system's behavior and failure/recovery scenarios. There are complexity tradeoffs--moving the cron scheduling outside of the container suddenly means you have moving parts outside of your container to test, instrument, monitor, and maintain. If you're already doing that, it's not too big of a deal, but if you aren't then I would think twice before adding all this complexity to your architecture, and make sure it's really worth it. (I can't tell from the article which approach works for their specific use cases...I'm pushing back against the general advice to "avoid cron in containers". I would default to using cron, and only move those processes to separate containers if you have a concrete reason to.)