4 ms·
For Python, structlog (https://www.structlog.org/en/stable/index.html https://www.structlog.org/en/stable/index.html) is a good tool for what you want. But I t
by itamarst 8y ago
For Python, structlog (https://www.structlog.org/en/stable/index.html https://www.structlog.org/en/stable/index.html) is a good tool for what you want.
But I think you can do better than structlog, with causal/trace-based logging. I built a Python library called Eliot to do so (https://eliot.readthedocs.io https://eliot.readthedocs.io).
You can use tracing in a single application just like you can for full distributed systems. This means each log message is part of a causal chain of actions, and you can see what caused it.
Log messages can still be viewed as individual disconnected messages, but you can also get a tree view (see https://eliot.readthedocs.io/en/stable/quickstart.html https://eliot.readthedocs.io/en/stable/quickstart.html for the code that resulted in these logs):
b1cb58cf-2c2f-45c0-92b2-838ac00b20cc
└── check_links/1 ⇒ started
├── timestamp: 2017-10-27 20:42:47.206684
├── urls:
│ ├── 0: http://eliot.readthedocs.io
│ └── 1: http://nosuchurl
├── download/2/1 ⇒ started
│ ├── timestamp: 2017-10-27 20:42:47.206933
│ ├── url: http://eliot.readthedocs.io
│ └── download/2/2 ⇒ succeeded
│ └── timestamp: 2017-10-27 20:42:47.439203
├── download/3/1 ⇒ started
│ ├── timestamp: 2017-10-27 20:42:47.439412
│ ├── url: http://nosuchurl
│ └── download/3/2 ⇒ failed
│ ├── errno: None
│ ├── exception: requests.exceptions.ConnectionError
│ ├── reason: HTTPConnectionPool(host='nosuchurl', port=80): Max retries exceeded with url: / (Caused by NewConnec…
│ └── timestamp: 2017-10-27 20:42:47.457133
└── check_links/4 ⇒ failed
├── exception: builtins.ValueError
├── reason: HTTPConnectionPool(host='nosuchurl', port=80): Max retries exceeded with url: / (Caused by NewConnec…
└── timestamp: 2017-10-27 20:42:47.457332
- photonios 8y agoI always thought about building something like this. Excellent work! Even serializes to JSON. Really nice. It would be nice to integrate it with Python's standard logging system similar to how structlog does this.
- itamarst 8y agoIt can take messages _from_ the standard logging library, and they even show up in the correct place in the tree: https://eliot.readthedocs.io/en/stable/generating/migrating.html https://eliot.readthedocs.io/en/stable/generating/migrating.... This is the preferable method in order to get the semantic model of tracing for all logs.
- anitil 8y agoThat is a fantastic library!