We're doing something similar but for our whole stack. We have a large, complex pipeline for processing incoming data. Historically it has been very opaque whi

We're doing something similar but for our whole stack. We have a large, complex pipeline for processing incoming data. Historically it has been very opaque which made debugging data errors hard. We added event logging throughout the pipeline. We are building a tool that feeds known data in one end and checks the final output. We can then use the event logging in between to monitor state throughout for errors. It also allows us to see how long it takes to transition between events. Ideally the tool will check every event for deviations from the expected output and alert us if any events fail. It will also alert us if the time between events rises above the average by a predefined threshold. I'd love to hear about any open source tools designed to do something like this.