3 ms·
I'm actually kinda surprised they went with SQLite here, log messages are the trivialest data format and there's no way you can't beat SQLite's speed by just no
by Spivak 2y ago
I'm actually kinda surprised they went with SQLite here, log messages are the trivialest data format and there's no way you can't beat SQLite's speed by just not having database logic in the middle at all. Just being able
to BYOAllocator for the logs themselves with such predictable linear memory usage would make this thing scream.
- dmd 2y agohttps://news.ycombinator.com/item?id=27897427 https://news.ycombinator.com/item?id=27897427
- WhyNotHugo 2y agoThe advantage of SQLite is being able to perform queries like “logs for service X and Y yesterday between 15hs and 16hs”.
- nolist_policy 2y agojournalctl -u ssh.service -u exim4.service --since='2024-09-14 15:00' --until='2024-09-14 16:00' (Systemd accepts e.g. 'yesterday' as a timestamp but not together with a time)
- cryptonector 2y agoYou're sort of right in that a B-tree is not a good data structure for logs given that append-only files are perfect for logs. But the point of using an RDBMS for logs is to be able to a) index the logs, b) provide a great search facility for logs. Perhaps a better design would be a virtual table plugin for SQLite3 that allows one to use log files as tables, then index and search them with SQLite3, but if one lacks the time to investigate that approach then oe can't be faulted for using SQLite3 directly.
- ivzhh 2y agoAgree, /var/log/messages is there for a long time, writing to log is never a problem. Digesting the log is the niche market and it is profitable enough that we have a lot of tools in this market (rotation, transmission, parsing, etc)