2 ms·

I don't really understand your list in view of the article. How is this solution more queryable than syslog: they record events into redis without any schema re

by cdavid 15y ago

I don't really understand your list in view of the article. How is this solution more queryable than syslog: they record events into redis without any schema related to it (just a string), so I fail to see the improvement there. They put it back to a log file anyway. It is not less or more centralized than syslog configured with centralization (which is trivial to set up). How is this more common than syslog across platforms (unless you include windows in "across platforms ?"). It is not simpler than syslog either, since writing to syslog is just a matter of using the right python logging backend. Analysing tons of data from syslog is a pain, but I don't see how any solution will not require at some point in the stack to enforce a format/structure in your log. How is this fundamentally different than post-processing http log ?

tptacek 15y ago
He's LPUSH'ing logs into a list key. Just by doing that one thing, he now has evented logging; he can subscribe to his raw logs with BRPOPLPUSH and without changing anything clientside start indexing them in better ways, or applying different policies to different logs. And Redis is easier to understand than syslog. We're pretending that there is zero friction to understanding syslog, as if any competent Unix person should automatically grok it. But syslog is a janky old rube goldberg machine and understanding it well pays off solely in the form of understanding a janky old rube goldberg machine that nobody will be using in 10 years.
- cdavid 15y ago
  I don't understand how BRPOPLPUSH helps you doing things you could not do with centralized syslog: you can also start indexing things with syslog by analysing the central logs without changing anything client-side. And in both cases, what will be needed is essentially the same: if the logs are not structured at the source, you will need to post-process them, the storage medium does not change any of that. The redis is easier than syslog is a bit of a strawman, because you will have to understand syslog anyway, since that's the only thing spoken by quite a few applications. In the OP'case, they are already using redis, so the cost on that side is very low, though.
- mihasya 15y ago
  The lady doth protest too much, no? The guy is using Redis as a buffer to what ultimately ends up just being a central log file. You know. rsyslog. None of the fancy things you have in mind. Which brings me to the same point someone's already made: using Redis buys you nothing, even if you had a more involved use case in mind. Saying that he can do something by subscribing to an event in Redis is sorta silly, isn't it? You could just as easily tail the logs, as many systems actually built for this purpose do. Once again, there are already things in *nix for this. The reality is, this is a janky, nubile solution to an already well-solved problem that is now getting thousands of views because of HN and antirez tweeting about it. Instead of learning the correct way of doing things, I bet a bunch of inexperienced developers are now going to say "BOY THIS IS SWELL" and cut themselves a nice fat chunk of technical debt for the not-that-far-off future.

by cdavid 15y ago

tptacek 15y ago

He's LPUSH'ing logs into a list key. Just by doing that one thing, he now has evented logging; he can subscribe to his raw logs with BRPOPLPUSH and without changing anything clientside start indexing them in better ways, or applying different policies to different logs. And Redis is easier to understand than syslog. We're pretending that there is zero friction to understanding syslog, as if any competent Unix person should automatically grok it. But syslog is a janky old rube goldberg machine and understanding it well pays off solely in the form of understanding a janky old rube goldberg machine that nobody will be using in 10 years.

cdavid 15y ago
I don't understand how BRPOPLPUSH helps you doing things you could not do with centralized syslog: you can also start indexing things with syslog by analysing the central logs without changing anything client-side. And in both cases, what will be needed is essentially the same: if the logs are not structured at the source, you will need to post-process them, the storage medium does not change any of that. The redis is easier than syslog is a bit of a strawman, because you will have to understand syslog anyway, since that's the only thing spoken by quite a few applications. In the OP'case, they are already using redis, so the cost on that side is very low, though.
mihasya 15y ago
The lady doth protest too much, no? The guy is using Redis as a buffer to what ultimately ends up just being a central log file. You know. rsyslog. None of the fancy things you have in mind. Which brings me to the same point someone's already made: using Redis buys you nothing, even if you had a more involved use case in mind. Saying that he can do something by subscribing to an event in Redis is sorta silly, isn't it? You could just as easily tail the logs, as many systems actually built for this purpose do. Once again, there are already things in *nix for this. The reality is, this is a janky, nubile solution to an already well-solved problem that is now getting thousands of views because of HN and antirez tweeting about it. Instead of learning the correct way of doing things, I bet a bunch of inexperienced developers are now going to say "BOY THIS IS SWELL" and cut themselves a nice fat chunk of technical debt for the not-that-far-off future.