4 ms·

Agreed. I stopped using Logstash for about a year and used it for a bit about a month ago. Awful experience. Awful documentation. Deprecated shit everywhere.

by bpchaps 11y ago

Agreed. I stopped using Logstash for about a year and used it for a bit about a month ago. Awful experience. Awful documentation. Deprecated shit everywhere. Inconsistent stackoverflow information and TWO external websites too help make logstash actually functional. Oh, and since Logstash is a Java based application - would it hurt to give some java stacktrace log parsing configs? Also, their shitty Debian repo management resulted in a bug that caused my company to lose $30,000. The world needs more ELK hate.

dozzie 11y ago
It's not Java, it's Ruby. Which is even worse, because distribution tarball with logstash weighs ridiculous 71MB (?!?) and requires JVM to run (?!?) (or at least nobody talks about it being runnable with MRI). > Also, their shitty Debian repo management resulted in a bug that caused my company to lose $30,000. Well, this is not their fault that much. If you had put any thought about using repositories, you wouldn't use random packages from random sources over which you have no control and no trust with regard to package retention policy or packages quality. Or maybe you would happily install also MongoDB from Mongo's site?
- vacri 11y ago
  Elasticsearch is Java, and the logstash tarball includes it and kibana from memory, so you can run it as an all-in-one where logstash launches it's own Elasticsearch. While I don't have the tiniest font in my terminal, I still couldn't read the entire Elasticsearch process line in htop, even when I'd stretched the terminal all the way across three monitors! The middle one was an ultrawide! I really wish Java would stop using arguments instead of storing config somewhere...
  - dozzie 11y ago
    > Elasticsearch is Java, and the logstash tarball includes it and kibana from memory, Not really. It's just logstash, along with some plugins (and what the heck are Maven bindings doing there?). ElasticSearch is another 29MB compressed, which is fine for a database-like thing, and Kibana 4.x takes 30MB compressed (150MB uncompressed, of which not the biggest part is Node.js copy).
- bpchaps 11y ago
  Got me on the ruby part. Meant to say JVM. My point was more to the fact that their site's incredibly short on examples. FWIW, I inherited a thrice-removed infrastructure in a latency-sensitive environment. It was fun ;) Logstash was installed on a couple machines the night before I was going to load some configs from my machine. I didn't load anything yet, except that logstash-web was running at 2s uptime. For some reason, (with upstart's help), the logstash installation included a broken webservice with a bad config. It got stuck in a JVM-birthing restart loop that was enough to cause significant impact. It happens. (Funny enough, same place used mongodb. We ran into an issue once where database cleanups failed once mongodb reached around 50% of the disk. Local copy cleanups are fun.)
- anentropic 11y ago
  it's JRuby: best of both worlds LOL
Rapzid 11y ago
My favorite thing about logstash was when the XML parser would choke and logstash would just quietly stop processing events.. I wish it was engineered as well as riemann.
- vacri 11y ago
  My favourite thing about logstash-forwarder was that it was apparently an actual intentional design decision to shut the service down if none of the log files it was watching moved in 24 hours. I'm still trying to figure that one out - a quiescent service... shuts itself down... by default... to what end? Ended up having to cronjob it to restart every day. If you wanted it not to do that, from memory you had to rebuild it from scratch. I've never actually come across that in a service before, one that shuts itself down due to inactivity. I mean, they must be out there, but I never would have guessed it'd be intentionally designed into a log shipper.
jedisct1 11y ago
As a replacement for Logstash, check out Flowgger: https://github.com/jedisct1/flowgger https://github.com/jedisct1/flowgger and Heka https://github.com/mozilla-services/heka https://github.com/mozilla-services/heka

by bpchaps 11y ago

dozzie 11y ago

It's not Java, it's Ruby. Which is even worse, because distribution tarball with logstash weighs ridiculous 71MB (?!?) and requires JVM to run (?!?) (or at least nobody talks about it being runnable with MRI). > Also, their shitty Debian repo management resulted in a bug that caused my company to lose $30,000. Well, this is not their fault that much. If you had put any thought about using repositories, you wouldn't use random packages from random sources over which you have no control and no trust with regard to package retention policy or packages quality. Or maybe you would happily install also MongoDB from Mongo's site?

vacri 11y ago
Elasticsearch is Java, and the logstash tarball includes it and kibana from memory, so you can run it as an all-in-one where logstash launches it's own Elasticsearch. While I don't have the tiniest font in my terminal, I still couldn't read the entire Elasticsearch process line in htop, even when I'd stretched the terminal all the way across three monitors! The middle one was an ultrawide! I really wish Java would stop using arguments instead of storing config somewhere...
- dozzie 11y ago
  > Elasticsearch is Java, and the logstash tarball includes it and kibana from memory, Not really. It's just logstash, along with some plugins (and what the heck are Maven bindings doing there?). ElasticSearch is another 29MB compressed, which is fine for a database-like thing, and Kibana 4.x takes 30MB compressed (150MB uncompressed, of which not the biggest part is Node.js copy).
bpchaps 11y ago
Got me on the ruby part. Meant to say JVM. My point was more to the fact that their site's incredibly short on examples. FWIW, I inherited a thrice-removed infrastructure in a latency-sensitive environment. It was fun ;) Logstash was installed on a couple machines the night before I was going to load some configs from my machine. I didn't load anything yet, except that logstash-web was running at 2s uptime. For some reason, (with upstart's help), the logstash installation included a broken webservice with a bad config. It got stuck in a JVM-birthing restart loop that was enough to cause significant impact. It happens. (Funny enough, same place used mongodb. We ran into an issue once where database cleanups failed once mongodb reached around 50% of the disk. Local copy cleanups are fun.)
anentropic 11y ago
it's JRuby: best of both worlds LOL

Rapzid 11y ago

My favorite thing about logstash was when the XML parser would choke and logstash would just quietly stop processing events.. I wish it was engineered as well as riemann.

vacri 11y ago
My favourite thing about logstash-forwarder was that it was apparently an actual intentional design decision to shut the service down if none of the log files it was watching moved in 24 hours. I'm still trying to figure that one out - a quiescent service... shuts itself down... by default... to what end? Ended up having to cronjob it to restart every day. If you wanted it not to do that, from memory you had to rebuild it from scratch. I've never actually come across that in a service before, one that shuts itself down due to inactivity. I mean, they must be out there, but I never would have guessed it'd be intentionally designed into a log shipper.

jedisct1 11y ago

As a replacement for Logstash, check out Flowgger: https://github.com/jedisct1/flowgger https://github.com/jedisct1/flowgger and Heka https://github.com/mozilla-services/heka https://github.com/mozilla-services/heka