4 ms·
I can understand they need to make money, but still a bit shady. Honestly though it's not something critical for my needs. Now if they could lower the resource
by nickserv 7y ago
I can understand they need to make money, but still a bit shady. Honestly though it's not something critical for my needs.
Now if they could lower the resource hogging a bit, that would interest me... Maybe even pay for that.
- jrockway 7y agoI use ELK for Kubernetes and network device logs, and I'm very much with you -- full text search is great, but it sure can be slow, even when running on $1000/month of AWS hardware. The conclusion that I've reached is that the whole lucene model for logs is kind of outdated. Why am I tuning Java GC params to run "grep foo /logs". I think computers today can do fine with sharded flat files, a minimal index ("which node contains logs from pod foo-2387438-2384738 at 12:34AM"), and then just scale horizontally over (log messages, searches). I hope my friends over at Tailscale are doing that and I can just move off ES entirely ;)
- ryanworl 7y agoIs Tailscale building a logging product?
- vorpalhex 7y agoELK stack user here - we actually found logstash to be our bottleneck. Changing it out for fluentd fixed our woes.
- nickserv 7y agoSame here, fluentd is much better, performance wise. But then I had to give ES more RAM because it couldn't take the hammering. In fact, increasing the throughput to ES was causing some pretty spectacular crashes, with the /var/log partition at 100% because of the verbosity of the dumps.
- majkinetor 7y agoLogstash sucks from both operational and developing perspective. I replaced it everywhere I could by sending structured logs directly from the app or by using newer integrated beats features.
- ReidZB 7y agoI believe Loki [1] is intended to basically run "grep foo" at scale (plus some extra niceties like labels). I haven't used it, but it seems interesting. [1]: https://grafana.com/loki https://grafana.com/loki