4 ms·

Fargate is pretty useful and it works well... when it works. One of the most annoying things is that you can't (as far as I know and I hope someone will correct

Fargate is pretty useful and it works well... when it works. One of the most annoying things is that you can't (as far as I know and I hope someone will correct me and show me the light) see the errors from the docker engine. This means that if there is some issue while spinning up your docker instances that you'd normally see in the docker engine's log, you won't know what is going on when using fargate. This makes debugging these issue a nightmare. Again, I hope I am just stupid and that there is some obvious way with which I CAN see the logs and someone will point out to me how.

mr-karan 7y ago
If your container feels to even run, then no logs are generated. Logs are only generated after your container spawns. So if you get errors in your `ENTRYPOINT` or `CMD` commands, the only way is to override these commands with something like [1] and debug why it's failing. [1]: `command: ["/bin/sh", "-c", "sleep 36000"]`
- zomglings 7y ago
  "tail -f /dev/null"
  - mosselman 7y ago
    That is a nice trick.
- mosselman 7y ago
  That is what I thought. How would your example command allow me to debug what is failing? Because the container will then spawn and allow me to enter it via ssh?
NathanKP 7y ago
I don't think I've ever seen an error from the "docker engine" on AWS Fargate. There is a task status field on the task, so if your container completely fails to start because you have a bad entrypoint command that can't be executed or your process immediately exits with a non zero exit code without outputting anything to stdout/stderr you will see a task status that says that the task stopped unexpectedly because the container exited, and why the container exited. If your containerized process does start and output to stdout/stderr, but then crashes for some reason you can see those stdout/stderr logs in CloudWatch, via the awslogs logging driver. In general I'd say the docker engine logs are the wrong place to try to diagnose an app running AWS Fargate. The point of AWS Fargate is to abstract away the Docker daemon, and just run your containers for you. So if the Docker engine itself was failing for some reason that is an issue for the AWS team on call to fix. Your area of responsibility starts with the application code that is running inside your container. The bits outside the container are the responsibility of an AWS employee.
- mosselman 7y ago
  In practice there are situations where you do want to see the error logs about your containers. I can't think of an example right now, but sometimes container configurations can be the cause of errors and seeing the logs is essential in not spending hours on investigating.
  - NathanKP 7y ago
    With AWS Fargate you can't interact directly with Docker in any way, you just use Fargate via an AWS API, and Fargate handles the rest. So we do validation in the task definition you supply via API to make sure that things check out before AWS Fargate launches the container. So you shouldn't see a "container configuration" based error because our API catches config issues first and won't let you create a task definition that is invalid. If there was such an error that caused Fargate to construct a bad container run command and fail to launch the container that would be a missing validation or bug inside our code and we'd fix that. If you do find something like this please reach out to me or open an AWS support ticket for the oncall team to look at, and we'll fix it!

mr-karan 7y ago

If your container feels to even run, then no logs are generated. Logs are only generated after your container spawns. So if you get errors in your `ENTRYPOINT` or `CMD` commands, the only way is to override these commands with something like [1] and debug why it's failing. [1]: `command: ["/bin/sh", "-c", "sleep 36000"]`

zomglings 7y ago
"tail -f /dev/null"
- mosselman 7y ago
  That is a nice trick.
mosselman 7y ago
That is what I thought. How would your example command allow me to debug what is failing? Because the container will then spawn and allow me to enter it via ssh?

NathanKP 7y ago

I don't think I've ever seen an error from the "docker engine" on AWS Fargate. There is a task status field on the task, so if your container completely fails to start because you have a bad entrypoint command that can't be executed or your process immediately exits with a non zero exit code without outputting anything to stdout/stderr you will see a task status that says that the task stopped unexpectedly because the container exited, and why the container exited. If your containerized process does start and output to stdout/stderr, but then crashes for some reason you can see those stdout/stderr logs in CloudWatch, via the awslogs logging driver. In general I'd say the docker engine logs are the wrong place to try to diagnose an app running AWS Fargate. The point of AWS Fargate is to abstract away the Docker daemon, and just run your containers for you. So if the Docker engine itself was failing for some reason that is an issue for the AWS team on call to fix. Your area of responsibility starts with the application code that is running inside your container. The bits outside the container are the responsibility of an AWS employee.

mosselman 7y ago
In practice there are situations where you do want to see the error logs about your containers. I can't think of an example right now, but sometimes container configurations can be the cause of errors and seeing the logs is essential in not spending hours on investigating.
- NathanKP 7y ago
  With AWS Fargate you can't interact directly with Docker in any way, you just use Fargate via an AWS API, and Fargate handles the rest. So we do validation in the task definition you supply via API to make sure that things check out before AWS Fargate launches the container. So you shouldn't see a "container configuration" based error because our API catches config issues first and won't let you create a task definition that is invalid. If there was such an error that caused Fargate to construct a bad container run command and fail to launch the container that would be a missing validation or bug inside our code and we'd fix that. If you do find something like this please reach out to me or open an AWS support ticket for the oncall team to look at, and we'll fix it!