6 ms·

If this is your advice then you shouldn't give advice. 1) 'docker logs' relies on using the json logdriver which means the log file is stored in /var/lib/docke

by TheDong 11y ago

If this is your advice then you shouldn't give advice. 1) 'docker logs' relies on using the json logdriver which means the log file is stored in /var/lib/docker/..... and grows forever. No rollover. No trimming. FOREVER. 2) What if your container dies? What if your host dies? Do you have any state at all or have you abstracted that out? Are your systems distributed 3) Always running does not answer finding where to run them 4) That only works if the container is running. What if it died? Also, docker logs is a fool's game 5) bingo, that's right at least 6) ....

kordless 11y ago
> If this is your advice then you shouldn't give advice. Stop with the blaming statements.
- toomuchtodo 11y ago
  Do you have a better way to say "your advice is bad. stop spreading misinformation"?
  - amouat 11y ago
    There's no need to say that at all. Just address the points and let everyone form their own opinions.
    - deleted 11y ago
      [deleted]
    - toomuchtodo 11y ago
      Because opinions are not the same as facts/best practices (nor is everyone's opinion equal). If you want to argue "this is the right way", be prepared to bring data and defend your statements. People here might be making critical decisions based on knowledge shared here, and they deserve the most accurate information possible.
      - amouat 11y ago
        Fully agree. That's why you should talk to the points and not make personal statements like "you shouldn't give advice".
      - joslin01 11y ago
        I agree, and would enjoy a thoughtful discussion as to why my stack doesn't work and why I'm an idiot. Instead, it's just been countered by 5 bland rebuttals and a debate about whether or not I am an idiot and if I should be referred to as such. Not sure what you're implying either, but you can tell in my comment, I never came out of the gates saying "This is the best way!"
      - collyw 11y ago
        In the world of Agile, opinions seem to be treated as fact.
  - gknoy 11y ago
    "There are some problems with your approach, and it could bite you in the tail when you least expect it. Here's what you should also know: ... (rest of GP's points)" This avoids saying "You're an idiot", which is nearly never constructive or helpful, and instead makes education and cooperation its goal. Most people respond better to that.
    - laichzeit0 11y ago
      Sometimes people need to know they're idiots without the person calling them an idiot having to tip-toe around the issue. Just like how some children are smarter than others and could actually benefit from ring labeled as such, put in a separate environment and given advanced problems to work on instead of lumping them with the rest of the herd. Just like fat people should be told they're fat and pay a premium for a seat on an airplane. Hell, if you look fat they should put you on a goddamn cattle scale and make you pay extra.
      - kordless 11y ago
        Without fully knowing what a person is thinking (and I'm talking in whole here), and the context they are thinking it in, it is impossible to always determine if what someone said is "idiotic" or not, much less be capable of determining if they themselves are an idiot. Given this provable fact, that means there exists a possibility someone ends up calling a savant an idiot for saying something that is viewed by them, or a group as a whole, as "idiotic".
        laichzeit0 11y ago
        And I'd wager to say that by and far the more likely possibility is that the person is actually an idiot. Just given that savants are an extremely small subset of the general population and if the bell curve is anything to go by, at least 50% of all people are below average intelligence (whether inherently stupid or uneducated doesn't matter, from a pragmatic point of view the result is the same)
        11y ago
  - kordless 11y ago
    Yeah, I do. Simply say what you find inaccurate about what they said and let everyone else come to their own conclusions. Saying "stop spreading misinformation" is basically speaking for others. Maybe we want to hear the misinformation because it's interesting or maybe you are wrong about it being informational or not. Either way, it's my choice to listen to him/her or not.
lucaspiller 11y ago
> 1) 'docker logs' relies on using the json logdriver which means the log file is stored in /var/lib/docker/..... and grows forever. No rollover. No trimming. FOREVER. Even without that issue, I'd prefer my logs to be centralised. So as well as my app should I be running a logging daemon, process monitoring, etc for each docker instance?
- hibikir 11y ago
  What we do at work is that we have our containers be in charge of talking 'out' on a given address and format for logs, and have things configured so that entire sets of machines end up speaking with the same log server (an ELK stack, in our case). The process monitoring is done per host: There are docker-aware tools that look at the host, and can peer into the container, to do this basic tracking. People are not kidding through, when they say that everything gets very complicated. All the things that we did by convention and manual configuration in regular VMs that are babysat manually have to be codified and automated. Docker is going to be a great ecosystem in 3 years, when the entire ecosystem matures. Today, it's the wild west, and you should only venture forth if having a big team of programmers and system administrators dedicated just to work on automation doesn't seem like a big deal.
- whitenoice 11y ago
  Similar to hibikir's reply, what we do is attach a volume container to all app containers and logs are written to that. The run elk stack to view parse logs. For process monitoring we run cAdvisor on each host to view the resource usage of each container. Since your apps are containerized it easy to monitor them for resource usage, hook it to nagios etc. We have built custom gui to do all this.
joslin01 11y ago
1 - Yes it does. This is a system problem more than a docker problem. For any relatively experienced engineer, they should be capable of realizing the logs must be stored somewhere and can plan around it. 2 - If a system dies and it has a state, then what do you do? If a dockerized process dies, and it has a state, then what do you do? This isn't some new problem to Docker. If my database service dies, you know what happens? It starts back up and connects to the persistent volume. Personally speaking, yes all of my services / systems are distributed. 3 - Most people don't need to start their services exactly at this point and then stop at another certain point (which is why I pretty much brushed over it). If they do, there's plenty of tools to do this that can also utilize docker. 4 - What if a system died? Does this mean you SSH'ing in isn't a viable option? (yes...) 5 - Yes, you love negativity so clearly this is your favorite 6 - ...? What? Do you have something more to say? It's cute that you like to poke holes and personally attack people, but really my comment was just how I go about things on a day-to-day basis. This is coming from someone who has 6 major Docker services abstracted out running all the time across 3 environments.. all capable of being updated via a `git push`. I think I have decent, practical advice to offer for other docker-minded practitioners and just decent advice to newcomers. Your grievances circle around logs not being centralized, easily-accessible (1, 2, 4).. You also don't outline any solutions yourself.
- vacri 11y ago
  > For any relatively experienced engineer, they should be capable of realizing the logs must be stored somewhere and can plan around it. Yes, the unrotated container logs are kept in a root-accessible-only location in a directory named after a long key that changes on every image restart - not conducive to manual log inspection, and definitely not conducive to centralised logging. That's not a 'system problem', it's Docker just being rude. Yes, a relatively experienced engineer can work around that... but why should they need to 'work around' it in the first place? Ironic really, that if you put a user in the 'docker' group, that they can do anything they want with the docker process, destroying as much data as they like or spinning up containers like nobody's business... but they can't see the container logfiles.
  - joslin01 11y ago

by TheDong 11y ago

kordless 11y ago

> If this is your advice then you shouldn't give advice. Stop with the blaming statements.

toomuchtodo 11y ago
Do you have a better way to say "your advice is bad. stop spreading misinformation"?
- amouat 11y ago
  There's no need to say that at all. Just address the points and let everyone form their own opinions.
  - deleted 11y ago
    [deleted]
  - toomuchtodo 11y ago
    Because opinions are not the same as facts/best practices (nor is everyone's opinion equal). If you want to argue "this is the right way", be prepared to bring data and defend your statements. People here might be making critical decisions based on knowledge shared here, and they deserve the most accurate information possible.
    - amouat 11y ago
      Fully agree. That's why you should talk to the points and not make personal statements like "you shouldn't give advice".
    - joslin01 11y ago
      I agree, and would enjoy a thoughtful discussion as to why my stack doesn't work and why I'm an idiot. Instead, it's just been countered by 5 bland rebuttals and a debate about whether or not I am an idiot and if I should be referred to as such. Not sure what you're implying either, but you can tell in my comment, I never came out of the gates saying "This is the best way!"
    - collyw 11y ago
      In the world of Agile, opinions seem to be treated as fact.
- gknoy 11y ago
  "There are some problems with your approach, and it could bite you in the tail when you least expect it. Here's what you should also know: ... (rest of GP's points)" This avoids saying "You're an idiot", which is nearly never constructive or helpful, and instead makes education and cooperation its goal. Most people respond better to that.
  - laichzeit0 11y ago
    Sometimes people need to know they're idiots without the person calling them an idiot having to tip-toe around the issue. Just like how some children are smarter than others and could actually benefit from ring labeled as such, put in a separate environment and given advanced problems to work on instead of lumping them with the rest of the herd. Just like fat people should be told they're fat and pay a premium for a seat on an airplane. Hell, if you look fat they should put you on a goddamn cattle scale and make you pay extra.
    - kordless 11y ago
      Without fully knowing what a person is thinking (and I'm talking in whole here), and the context they are thinking it in, it is impossible to always determine if what someone said is "idiotic" or not, much less be capable of determining if they themselves are an idiot. Given this provable fact, that means there exists a possibility someone ends up calling a savant an idiot for saying something that is viewed by them, or a group as a whole, as "idiotic".
      - laichzeit0 11y ago
        And I'd wager to say that by and far the more likely possibility is that the person is actually an idiot. Just given that savants are an extremely small subset of the general population and if the bell curve is anything to go by, at least 50% of all people are below average intelligence (whether inherently stupid or uneducated doesn't matter, from a pragmatic point of view the result is the same)
        11y ago
- kordless 11y ago
  Yeah, I do. Simply say what you find inaccurate about what they said and let everyone else come to their own conclusions. Saying "stop spreading misinformation" is basically speaking for others. Maybe we want to hear the misinformation because it's interesting or maybe you are wrong about it being informational or not. Either way, it's my choice to listen to him/her or not.

lucaspiller 11y ago

> 1) 'docker logs' relies on using the json logdriver which means the log file is stored in /var/lib/docker/..... and grows forever. No rollover. No trimming. FOREVER. Even without that issue, I'd prefer my logs to be centralised. So as well as my app should I be running a logging daemon, process monitoring, etc for each docker instance?

hibikir 11y ago
What we do at work is that we have our containers be in charge of talking 'out' on a given address and format for logs, and have things configured so that entire sets of machines end up speaking with the same log server (an ELK stack, in our case). The process monitoring is done per host: There are docker-aware tools that look at the host, and can peer into the container, to do this basic tracking. People are not kidding through, when they say that everything gets very complicated. All the things that we did by convention and manual configuration in regular VMs that are babysat manually have to be codified and automated. Docker is going to be a great ecosystem in 3 years, when the entire ecosystem matures. Today, it's the wild west, and you should only venture forth if having a big team of programmers and system administrators dedicated just to work on automation doesn't seem like a big deal.
whitenoice 11y ago
Similar to hibikir's reply, what we do is attach a volume container to all app containers and logs are written to that. The run elk stack to view parse logs. For process monitoring we run cAdvisor on each host to view the resource usage of each container. Since your apps are containerized it easy to monitor them for resource usage, hook it to nagios etc. We have built custom gui to do all this.

joslin01 11y ago

1 - Yes it does. This is a system problem more than a docker problem. For any relatively experienced engineer, they should be capable of realizing the logs must be stored somewhere and can plan around it. 2 - If a system dies and it has a state, then what do you do? If a dockerized process dies, and it has a state, then what do you do? This isn't some new problem to Docker. If my database service dies, you know what happens? It starts back up and connects to the persistent volume. Personally speaking, yes all of my services / systems are distributed. 3 - Most people don't need to start their services exactly at this point and then stop at another certain point (which is why I pretty much brushed over it). If they do, there's plenty of tools to do this that can also utilize docker. 4 - What if a system died? Does this mean you SSH'ing in isn't a viable option? (yes...) 5 - Yes, you love negativity so clearly this is your favorite 6 - ...? What? Do you have something more to say? It's cute that you like to poke holes and personally attack people, but really my comment was just how I go about things on a day-to-day basis. This is coming from someone who has 6 major Docker services abstracted out running all the time across 3 environments.. all capable of being updated via a `git push`. I think I have decent, practical advice to offer for other docker-minded practitioners and just decent advice to newcomers. Your grievances circle around logs not being centralized, easily-accessible (1, 2, 4).. You also don't outline any solutions yourself.

vacri 11y ago
> For any relatively experienced engineer, they should be capable of realizing the logs must be stored somewhere and can plan around it. Yes, the unrotated container logs are kept in a root-accessible-only location in a directory named after a long key that changes on every image restart - not conducive to manual log inspection, and definitely not conducive to centralised logging. That's not a 'system problem', it's Docker just being rude. Yes, a relatively experienced engineer can work around that... but why should they need to 'work around' it in the first place? Ironic really, that if you put a user in the 'docker' group, that they can do anything they want with the docker process, destroying as much data as they like or spinning up containers like nobody's business... but they can't see the container logfiles.
- joslin01 11y ago