4 ms·
Good luck with that searching, and remember to share with HN. I find ElasticSearch somewhat brittle (I need to restart it every few weeks or so, because it sto
by dozzie 11y ago
Good luck with that searching, and remember to share with HN.
I find ElasticSearch somewhat brittle (I need to restart it every few weeks or
so, because it stops accepting any data or queries), and I really do want to
replace it, since it's a memory hog (no data, and it already needs 230MB RAM),
but I haven't found any sensible log storage yet. All I got is this document
searching engine.
- gibrown 11y ago230MB of Heap is very small for most ES usages, and is likely the reason for you needing to restart it often. I run some very large clusters and even with older versions of ES have seen them run billions of indexing and query operations without problems for 4-6 months without any restarts.
- dozzie 11y ago> 230MB of Heap is very small for most ES usages 230MB RSS, and this is not "ES usage", it's just after starting it with no data whatsoever. I've seen databases which use less memory than that when working. > I run some very large clusters [...] without problems for 4-6 months without any restarts. Well, good for you. Being a sysadmin, I understand very well that my ES server may be a specimen combining bad kernel compilation, broken JVM version, unstable ES release, and ground water emanating bad energy, so I don't complain much. Still, I would be very, very happy to see any alternative for storing JSONified logs.
- packetized 11y agoDespite what they recommend, I'd move to G1GC over CMS. Did wonders for our stability. This is on ES2.1.0 & OpenJDK 1.8.0-u80, IIRC.
- Jgrubb 11y agoThat was my experience exactly. The value of being able to search through ours logs was immediately apparent, so when it kept tanking every couple weeks it was fairly easy to talk my boss into paying a tiny bit more for Loggly. AFAICT Loggly is the ELK stack with a different theme applied to the UI, and I don't have to worry that it's down when I need it. The preceding has been an unpaid endorsement for Loggly.
- dozzie 11y agoI would prefer much more to have something hosted by me and being open source at the same time (I'm looking at you, Splunk). I'm desperate enough to write such thing myself (and I'm already doing some research on indexing semi-structured data), even if it would turn out not-too-scalable (I only have ~1.5GB/day of uncompressed logs), but I'm scared by the need of tolerance for abrupt shutdowns and hitting a free disk space limit, i.e., database recoverability.
- Jgrubb 11y agoHey, I looked at Splunk in the last few days as well, but it seems really expensive once you want to get beyond 500MB a day. They don't have an OSS "community" version, do they (?) because I totally feel you on that. Basically I need ELK, but I need it to be a lot more stable and documented better.
- packetized 11y agoLoggly is a five digit monthly bill at our present logging volume. No thanks.
- asafm 11y agoI suggest taking a look at the logz.io - it's essentially ELK as a service. We take care of many problems ES has and also have great features extra such as Alerting, Apps and Insights. I'm one of the core developers there so I'm fluent in pricing but last time I checked it's wasn't expensive and definitely worth your time and hosting costs. One the the great things about it is that you can simply migrate your dashboard and just ship to our Url.
- Jgrubb 11y agoWhat are you using? I'd rather run it myself, but like I said ELK was a bit too bleeding edge for me in the stability dept.
- packetized 11y ago
- packetized 11y agoI'm looking at Graylog right now.
- rpedela 11y agoWhich version? This reason I ask is that 2.x is a lot more stable for me than previous versions. I have to restart much less often and I haven't had ES "forget" my index since upgrading.
- dozzie 11y agoSome 1.x, and I don't intend to upgrade to 2.x, as it requires Kibana upgrade, and I don't want to deal with 150MB Node.js insanity Kibana 4 brings.