6 ms·
This is the latest in a long string of irritations that will have me looking for a replacement logging datastore post-haste. I've already replaced Logstash with
by packetized 11y ago
This is the latest in a long string of irritations that will have me looking for a replacement logging datastore post-haste. I've already replaced Logstash with Heka; I guess ES is next.
- ktamura 11y agoCurious if you had a chance to look at Fluentd (disclaimer: I'm one of the maintainers of Fluentd) If you did, I'm interested in your honest feedback.
- packetized 11y agoNot yet, but I'll give it a shot. How's sharding/scalability?
- packetized 11y agoWe already have a transport pipeline; what's needed is a decently scalable, searchable, easy-to-use datastore for short/medium/long-term log analysis.
- zenlikethat 11y agoRight? An ES replacement in Go would be so great.
- sz4kerto 11y agoWhy do you care of the language?
- glial 11y agoFor one thing it means you wouldn't have to screw around with the peculiarities of the JVM.
- zenlikethat 11y agoRidiculously easy deployment and lower memory footprint.
- alexott 11y agoIs there already Lucene for Go? ;-)
- vruiz 11y agohttp://www.blevesearch.com/ http://www.blevesearch.com/ It has an very long way ahead to catch up with Lucene, but it's a very promising start.
- rakoo 11y agoNaïve question from a bystander: how much better is Lucene compared to bleve ?
- eternalban 11y ago/aside: I get rewriting for the n-th time some utility (say a db driver) in language du jour, but (possibly poorly) replicating things like Lucene never made sense. Lucene is a fairly solid tech by all accounts.
- dozzie 11y agoGood luck with that searching, and remember to share with HN. I find ElasticSearch somewhat brittle (I need to restart it every few weeks or so, because it stops accepting any data or queries), and I really do want to replace it, since it's a memory hog (no data, and it already needs 230MB RAM), but I haven't found any sensible log storage yet. All I got is this document searching engine.
- gibrown 11y ago230MB of Heap is very small for most ES usages, and is likely the reason for you needing to restart it often. I run some very large clusters and even with older versions of ES have seen them run billions of indexing and query operations without problems for 4-6 months without any restarts.
- dozzie 11y ago> 230MB of Heap is very small for most ES usages 230MB RSS, and this is not "ES usage", it's just after starting it with no data whatsoever. I've seen databases which use less memory than that when working. > I run some very large clusters [...] without problems for 4-6 months without any restarts. Well, good for you. Being a sysadmin, I understand very well that my ES server may be a specimen combining bad kernel compilation, broken JVM version, unstable ES release, and ground water emanating bad energy, so I don't complain much. Still, I would be very, very happy to see any alternative for storing JSONified logs.
- packetized 11y agoDespite what they recommend, I'd move to G1GC over CMS. Did wonders for our stability. This is on ES2.1.0 & OpenJDK 1.8.0-u80, IIRC.
- Jgrubb 11y agoThat was my experience exactly. The value of being able to search through ours logs was immediately apparent, so when it kept tanking every couple weeks it was fairly easy to talk my boss into paying a tiny bit more for Loggly. AFAICT Loggly is the ELK stack with a different theme applied to the UI, and I don't have to worry that it's down when I need it. The preceding has been an unpaid endorsement for Loggly.