3 ms·
In a way, it's a subset of the distributed systems tracing problem - you have multiple tasks running in parallel on the same node, but they will have been initi
by btown 3y ago
In a way, it's a subset of the distributed systems tracing problem - you have multiple tasks running in parallel on the same node, but they will have been initiated as different (sub) tasks, and should be tracked by the specific task via which they were initiated. So systems like OpenTelemetry and Honeycomb can be great for this, allowing you to see events in aggregate as well as in the context of a trace that propagates between different threads and systems.
https://opentelemetry.io/docs/languages/python/getting-started/ https://opentelemetry.io/docs/languages/python/getting-start...
https://docs.honeycomb.io/getting-data-in/opentelemetry/python-distro/#distributed-trace-propagation https://docs.honeycomb.io/getting-data-in/opentelemetry/pyth...
But there's so much complexity there that IMO it's best left outside of standard libraries - and it's indeed a daunting amount of new vocabulary for newcomers. I'm not aware of simpler abstractions on top of the broader telemetry ecosystem for monitoring simple parallelization, but arguably there should be one that keeps things quite simple.