3 ms·
Although I agree in principle one other aspect that becomes critical in high volume systems is efficiency. Logging is generally a text stream and in production
by nodefortytwo 7y ago
Although I agree in principle one other aspect that becomes critical in high volume systems is efficiency.
Logging is generally a text stream and in production should only be warning+ levels (imo) so the impact on performance is negligible and really only when something is going wrong.
I want metrics all of the time, potentially 10 or 20 metrics per request/action, a high performance, low network method for sending those metrics with low latency is critical.
We need both systems and both should be treated as tier 1 systems within an organisation. I don't think pushing metrics into a log stream is a scalable architecture.
- delusional 7y agoBeing a text stream, you'd also have to reparse that text stream, and hope that the text stream never changes. Which is at odds with how I usually use logs (as places to dump information for humans).
- yehosef 7y agoNowadays using something like elasticsearch is common for this and then you don't have to reparse it. It's already indexed and you can search for what you want. If you're going back to the raw text files, then you're right. But I'm not sure why anyone would do that these days.
- haddr 7y agobut then it might be expensive to encode every possible dimension in the log stream
- blondin 7y agothe author is saying the same thing. although he didn't mention efficiency in his article, which is an excellent remark btw. but he is saying you can usually go from logs to metrics but not the other way around. and that we often need both.