5 ms·
Fwiw this is how I use Loki most of the time. Pick an app label, pick a time period, look at raw logs. The LogQL for this ends up something like `{app="workload
by pluies 2y ago
Fwiw this is how I use Loki most of the time. Pick an app label, pick a time period, look at raw logs. The LogQL for this ends up something like `{app="workload-foo"}`. Loki is excellent at that.
Then if I know which pod I'll filter down to a specific pod with `{pod="workload-foo-1234"}`, sometimes I'll search for a specific term (error message etc) with `{pod="workload-foo-1234"} |= "error message"` then look at the logs around that. There's really no point writing complicated queries unless you need to.
- School-Cotton 2y agoThat will, if I understand correctly, get the logs for one pod, not for one process. For example if the pod restarted 10 times you will not get 10 separate files from that query.
- sofixa 2y agoYou'd have the label shown in the output that indicates the log line in question is from a different process/pod/container/host/whatever.
- School-Cotton 2y agoHow so? The pod, container, and host labels should be the same for a process that crashes and is automatically restarted, no?
- liampulles 2y agoEven more than that, if you are running multiple instances of the app in multiple pods concurrently, then all of those logs will be joined together.
- remram 2y agoThat's definitely false
- Matthias247 2y agoWhy? If the pod is defined to spawn multiple containers, and each container runs the same application, then this seems true to me? Unless you would add an additional filter on the container name.
- remram 2y agoWell yes obviously you have to filter on container if you want a single container (just like kubectl logs -l <...>). The parent comment was phrased as a limitation of Loki, of course if you request all logs for an application you'll get all containers, or if you request all logs for an applications or a namespace you will get that. Not being able to filter between multiple processes or multiple restart of a container was a genuine issue, not being able to filter between pods of a deployment is not.
- Matthias247 2y agoI actually didn't understand it being phrased as a limitation. It could also be a feature - maybe one would prefer to look at logs for multiple services within a single query? Anyhow, the nice thing about the system is that one can get anything that is preferable as long as the logs are annotated correctly (with pod and container id).
- remram 2y agoNo, once again, the trouble is that you can't get the logs for a specific execution. If a container in your pod restarts, that is invisible to Loki, you have to look for whatever the container writes on startup and cut there manually. If you want a specific process in your container, it's mixed with the rest.
- liampulles 2y agoIts certainly true in my environment, maybe not others though? Apologies!
- nebster 2y agoI'm not sure I really understand this. If you mean one instance in each pod, then each should be labelled differently and you can filter down to one instance. If you mean running multiple instances in each pod (and container?), then the standard kubectl log output will also have them all joined together. For both of those, you would need to add another unique identifier to each line, or run each instance in a separate container so you can submit the logs with the pod name and container name combined being the unique identifier.