6 ms·

Bootstrap Based UI for Logstash (Open Source Splunk)

99 points by kordless 14y ago

kordless 14y ago
A working demo is here: http://urly.stackgeek.com/C2v http://urly.stackgeek.com/C2v. Author of Logstash is here: http://semicomplete.com http://semicomplete.com.
rurounijones 14y ago
Currently experimenting with logstash and Kibana on internal systems and very happy so far, no complaints with it. Be warned though that logstash is not mature software, get on the mailing list and read the github page. Now if only the rails logging system wasn't so tightly integrated and string-happy.
- kordless 14y ago
  I'm pretty sure he's working on it fulltime as of a month ago or so.
  - josegonzalez 14y ago
    whack (author of logstash) works at dreamhost now, and is indeed working on logstash full-time. The kibana ui is, afaik, being ported to ruby atm.
  - rurounijones 14y ago
    Groovy, didn't know that. When I said "not mature" I just meant in terms of number of deployments and age of the code-base. No knock against author intended.
    - lusis 14y ago
      You might underestimate the size of deployments of logstash. Mailchimp runs logstash in a pretty sizeable cluster for all traffic coming into HTTP front-ends. I can think of quite a few sizeable logstash installs that I can't mention.
cnlwsu 14y ago
Looks similar to http://graylog2.org/ http://graylog2.org/
- lusis 14y ago
  Logstash and Graylog are complimentary. Most people, myself included, we're originally using Graylog2 in conjunction with Logstash. Graylog2, though, had problems with it's original implementation based on capped containers in MongoDB. It has since moved to ElasticSearch. There are both gelf inputs and outputs for Logstash so you can send your logs to Logstash as if they were going to Graylog2 and do additional munging and still send them out to Graylog2 from there.
  - cmer 14y ago
    Would you mind going into more details as to how they are similar and what the differences are between the two? Should I run both, or pick one? You seem to suggest to run both in tandem but I'm not sure I see why since at first sight they seemed pretty much the same to me. I was planning to setup Logstash next week so your input would really help. Thanks!
    - lusis 14y ago
      Graylog2 only handles syslog and its own protocol (GELF) for accepting log events. Graylog2 uses ElasticSearch for data storage. Logstash can use ElasticSearch or just send the data elsewhere. The Graylog2 web interface is pretty awesome and it has some neat stuff built in. Logstash ships with a fairly spartan web interface though we're going to replace it with a ruby port of Kibana in the future. Logstash can accept data from GELF senders (via the gelf input plugin I wrote) or send to gelf receivers (like Graylog2). Graylog2 is awesome, don't get me wrong. I just had to bail on it when it was still on MongoDB because I couldn't justify the cost of instances needed to get a MongoDB instance that could hold more than 4 hours of data.
northisup 14y ago
Why on earth does being made with bootstrap matter for this? The headline should read "Logstash UI that doesn't suck" What something is built with doesn't matter, that it works matters.
- rapind 14y ago
  I don't see a problem with it TBH. I pretty much knew exactly what to expect because of this wording, which is a good thing.
  - kordless 14y ago
    That's what I was shooting for! :)
lusis 14y ago
Please don't call logstash "an open source splunk". It's no such thing. Splunk still has features that logstash doesn't have (yet). Logstash has quite a few features that Splunk doesn't have. Jordan had never seen (or to my knowledge has yet to see) splunk at all. I don't know about Pete. Myself, I haven't used Splunk since trying a very early release once in the very first days of it. Point being, Logstash doesn't call itself an "open source splunk". In fact I've considered adding an output to SplunkStorm to Logstash. Do I think Logstash is better? Yep. Do I know people who swear by Splunk? Yep. Competition is healthy.
- drone 14y ago
  I agree, "an open-source X" implies it re-implements X. LogStash is a log management system, which is one application of Splunk. (There are a lot of players in this space.) And, much like Splunk, it seems to be well-fit for users who prefer to get down to the nuts and bolts. I haven't tried it yet, but I don't have a need for real LM or IT search these days, when I do - it'll be in my list of things to set up and try. I like what I've seen, but I don't see much IT search or automation here. Disclaimer: I was the architect of a closed-source competitor to Splunk in the log management space.
- kordless 14y ago
  The commonality is they both ingest logs and provide fulltext search for said logs. That's enough to loosely comare the two for purposes of promotion here on HN, or even getting good mentions on ServerFault: http://serverfault.com/questions/62687/alternatives-to-splunk http://serverfault.com/questions/62687/alternatives-to-splun... I'm certainly not the first to make this comparison.
  - adient 14y ago
    Although logstash does have a built in elasticsearch, I wouldn't really say anyone uses logstash itself to provide search for the logs. Logstash itself just provides a way to move events from one place to another, that's all.
    - 14y ago
superdude 14y ago
You need to add some CSS to account for extra pixels the responsive navbar uses. Also, the JavaScript for the navbar menu does does not seem to be working.
tavishmctavish 14y ago
Whats Splunk?
- sciurus 14y ago
  http://en.wikipedia.org/wiki/Splunk http://en.wikipedia.org/wiki/Splunk
PanMan 14y ago
Could this be used on Elasticsearch in general? Any pointers what's needed for that? The interface looks nice, and we have a large(ish) Elasticsearch cluster I'd like to try this on.

99 points by kordless 14y ago

A working demo is here: http://urly.stackgeek.com/C2v http://urly.stackgeek.com/C2v. Author of Logstash is here: http://semicomplete.com http://semicomplete.com.

rurounijones 14y ago

Currently experimenting with logstash and Kibana on internal systems and very happy so far, no complaints with it. Be warned though that logstash is not mature software, get on the mailing list and read the github page. Now if only the rails logging system wasn't so tightly integrated and string-happy.

kordless 14y ago
I'm pretty sure he's working on it fulltime as of a month ago or so.
- josegonzalez 14y ago
  whack (author of logstash) works at dreamhost now, and is indeed working on logstash full-time. The kibana ui is, afaik, being ported to ruby atm.
- rurounijones 14y ago
  Groovy, didn't know that. When I said "not mature" I just meant in terms of number of deployments and age of the code-base. No knock against author intended.
  - lusis 14y ago
    You might underestimate the size of deployments of logstash. Mailchimp runs logstash in a pretty sizeable cluster for all traffic coming into HTTP front-ends. I can think of quite a few sizeable logstash installs that I can't mention.

cnlwsu 14y ago

Looks similar to http://graylog2.org/ http://graylog2.org/

lusis 14y ago
Logstash and Graylog are complimentary. Most people, myself included, we're originally using Graylog2 in conjunction with Logstash. Graylog2, though, had problems with it's original implementation based on capped containers in MongoDB. It has since moved to ElasticSearch. There are both gelf inputs and outputs for Logstash so you can send your logs to Logstash as if they were going to Graylog2 and do additional munging and still send them out to Graylog2 from there.
- cmer 14y ago
  Would you mind going into more details as to how they are similar and what the differences are between the two? Should I run both, or pick one? You seem to suggest to run both in tandem but I'm not sure I see why since at first sight they seemed pretty much the same to me. I was planning to setup Logstash next week so your input would really help. Thanks!
  - lusis 14y ago
    Graylog2 only handles syslog and its own protocol (GELF) for accepting log events. Graylog2 uses ElasticSearch for data storage. Logstash can use ElasticSearch or just send the data elsewhere. The Graylog2 web interface is pretty awesome and it has some neat stuff built in. Logstash ships with a fairly spartan web interface though we're going to replace it with a ruby port of Kibana in the future. Logstash can accept data from GELF senders (via the gelf input plugin I wrote) or send to gelf receivers (like Graylog2). Graylog2 is awesome, don't get me wrong. I just had to bail on it when it was still on MongoDB because I couldn't justify the cost of instances needed to get a MongoDB instance that could hold more than 4 hours of data.

northisup 14y ago

Why on earth does being made with bootstrap matter for this? The headline should read "Logstash UI that doesn't suck" What something is built with doesn't matter, that it works matters.

rapind 14y ago
I don't see a problem with it TBH. I pretty much knew exactly what to expect because of this wording, which is a good thing.
- kordless 14y ago
  That's what I was shooting for! :)

lusis 14y ago

Please don't call logstash "an open source splunk". It's no such thing. Splunk still has features that logstash doesn't have (yet). Logstash has quite a few features that Splunk doesn't have. Jordan had never seen (or to my knowledge has yet to see) splunk at all. I don't know about Pete. Myself, I haven't used Splunk since trying a very early release once in the very first days of it. Point being, Logstash doesn't call itself an "open source splunk". In fact I've considered adding an output to SplunkStorm to Logstash. Do I think Logstash is better? Yep. Do I know people who swear by Splunk? Yep. Competition is healthy.

drone 14y ago
I agree, "an open-source X" implies it re-implements X. LogStash is a log management system, which is one application of Splunk. (There are a lot of players in this space.) And, much like Splunk, it seems to be well-fit for users who prefer to get down to the nuts and bolts. I haven't tried it yet, but I don't have a need for real LM or IT search these days, when I do - it'll be in my list of things to set up and try. I like what I've seen, but I don't see much IT search or automation here. Disclaimer: I was the architect of a closed-source competitor to Splunk in the log management space.
kordless 14y ago
The commonality is they both ingest logs and provide fulltext search for said logs. That's enough to loosely comare the two for purposes of promotion here on HN, or even getting good mentions on ServerFault: http://serverfault.com/questions/62687/alternatives-to-splunk http://serverfault.com/questions/62687/alternatives-to-splun... I'm certainly not the first to make this comparison.
- adient 14y ago
  Although logstash does have a built in elasticsearch, I wouldn't really say anyone uses logstash itself to provide search for the logs. Logstash itself just provides a way to move events from one place to another, that's all.
  - 14y ago

superdude 14y ago

You need to add some CSS to account for extra pixels the responsive navbar uses. Also, the JavaScript for the navbar menu does does not seem to be working.

tavishmctavish 14y ago

Whats Splunk?

sciurus 14y ago
http://en.wikipedia.org/wiki/Splunk http://en.wikipedia.org/wiki/Splunk

PanMan 14y ago

Could this be used on Elasticsearch in general? Any pointers what's needed for that? The interface looks nice, and we have a large(ish) Elasticsearch cluster I'd like to try this on.