3 ms·
Due to the nature of our task-monitoring service at Cronitor, missing a ping from a user could result in a false-positive alert. Needless to say, it's bad news.
by encoderer 11y ago
Due to the nature of our task-monitoring service at Cronitor, missing a ping from a user could result in a false-positive alert. Needless to say, it's bad news. As a failsafe against exactly the situation described here, we run a daemon on every webserver that tails the nginx error log and injects failed messages into SQS. These at-least-once semantics work for us because messages that somehow get delivered twice are idempotent. Somedays I think this is a kludge, other days I'm pleased that it saves our butt (including, when SQS is unavailable or slow which thankfully rarely happens, but rarely is not never).