one thing is missing in the list: a time machine - the system should capture verbose information about events occuring before the error - all these DEBUG/INFO m

one thing is missing in the list: a time machine - the system should capture verbose information about events occuring before the error - all these DEBUG/INFO messages you usually disable in production. Without that you have to change the log level and try to repeat the error condition. And dont go too far, or you'll end up with a cluster of log aggregation machines just for collecting the logs from one application. Like 'logstash/kibana' behemoth ;)