3 ms·
Ask HN: Custom logging for web applications
I have an application (written in Django) that needs to log lots of data (visitor stats from a Flash app, a url is pinged whenever users do certain things). I've rolled by own logging system along with a library for extracting statistics from the logged data based on the map/reduce pattern. However, I'm having performance problems with my initial quick and dirty implementation (in the stats generation, not in the logging bit yet). This seems like a problem that must have been solved plenty of times before. What do people use for this sort of stuff?
- bayareaguy 18y agoI'd suggest you send all your custom stuff to syslog so that you can easily consolidate events however you like and then work with your admin to deploy whatever variant of syslog best fits your requirements. Personally I think nsyslog is good. Some others are listed here: http://www.loganalysis.org/sections/syslog/syslog-replacements http://www.loganalysis.org/sections/syslog/syslog-replacemen...
- rarrrrrr 18y agoA few months ago we actually created a system that does exactly this, plugging into the Ganglia data visualization system many sites use. We have bindings for Perl and Python. Your app code just emits non-blocking UDP packets to a local collector daemon and goes on with its business. GPLv3 code here: https://spideroak.com/code https://spideroak.com/code (StatGrabber) "If you can't measure it, you can't manage it." It's really nice to see graphs of the business or application specific stuff (signups, connected users, revenue, anything that's event driven) in parallel with the regular system graphs of things like bandwidth, memory, and so on. When you look at the weekly or monthly graphs usage trends and bottlenecks become forehead-smacking obvious. Ganglia demo: http://monitor.millennium.berkeley.edu/ http://monitor.millennium.berkeley.edu/