5 ms·
I see your idea but you could drop the JSON and use rsyslogd + logrotate + grep? You can grep 10 gig files on a $5 VPS easily and quickly! I can't speak for a $
by unxdfa 4y ago
I see your idea but you could drop the JSON and use rsyslogd + logrotate + grep? You can grep 10 gig files on a $5 VPS easily and quickly! I can't speak for a $4 one ;)
- tiagod 4y agoIf you use grep you'll be doing the same expensive operation every time, following files naively will fail after rotation, etc... And if you use something like Loki it's easier to integrate with other tools to react to the logs
- unxdfa 4y agoIt’s potentially a premature optimisation to not do that expensive operation every time. Loki and brethren have a significant infrastructure cost and cognitive load to consider. I speak from experience and know where the ROI appears and it’s far from the use case specified here.
- ilyt 4y ago> following files naively will fail after rotation, ...so what you're saying they have to write "tail -F" instead of "tail". > If you use grep you'll be doing the same expensive operation every time if you have ingest that low it barely matters. Modern grep replacements are pretty fast
- mekster 4y agoWhy do people like to stick to inefficient ancient method like grep for log viewing? Try tools like Metabase and see how it makes your log reading far better.