5 ms·
i’ve been using this in production to stream logs to a web console (~2,000 sets of logs distributed across 4 server with about 350 active connections at a time)
by rileytg 8y ago
i’ve been using this in production to stream logs to a web console (~2,000 sets of logs distributed across 4 server with about 350 active connections at a time) and have never had any issues
- hnarn 8y agoHow do you know that you haven't had any issues?
- perlgeek 8y agoUsually you notice issues by the server locking up, the clients reporting problems, or messages being missing from the log (which you tend to notice when you search for specific things in the logs).
- hnarn 8y agoSo how do you know that you're not just getting 80% of "log entry X" just because you get hits for them every now and then?
- perlgeek 8y agoLogging is usually pretty deterministic. A request might generate log entries a, b1, c or a, b2, c, depending on some conditions. The exact contents vary by request (otherwise there would be no need to log them), but the type is always the same. If you find logs for c without either b1/b2 or a, you know log entries went missing. If you have a 20% miss rate with recording your log entries, and you analyze just 20 log entries, the chance that one of them is missing is already around 98.8%. If you actually use your logs for anything, it becomes pretty obvious pretty quickly when they are incomplete.
- rileytg 8y agowe track errors both server and client side. only errors we routinely see are network issues.
- rileytg 8y agoto be fair there could be silent failures, but after 4 years of daily use by people who are experts on these logs, we would have had at least a couple reports of missing lines. we often download the files post viewing on streams also. mostly it’s exceptions so we’d easily notice missing lines in stacks