5 ms·
You don't want to use sqlite for logs -- it is way too complex, and in particular, it may roll back (=lose log entries) if it is low on space, and permanently d
by theamk 7y ago
You don't want to use sqlite for logs -- it is way too complex, and in particular, it may roll back (=lose log entries) if it is low on space, and permanently damage data when writing onto disks with errors. It also makes wrong performance tradeoffs for logging -- your choices are either sync() all the time, or roll back on unexpected termination.
I don't like the that journald goes backwards in space and updates indices -- I like "append-only" attribute -- but at least it writes data sequentially one after another, which makes it pretty tolerant of data loss.
As for "without escape hatch", I don't remember any functionality like that? In fact, systemd is full of escape hatches which are documented and supported. Just recently, I had to modify "is computer online" check -- it was amazingly easy, all I had to do was to drop-in a fragment file for the right service. I remember doing the same in upstart -- it was a total pain, requiring reading the source code spread over dozens of undocumented files.
- realusername 7y agoSqlite is battle-tested though, it has one of the most comprehensive test suite and is used on billions of devices already. What systemd is doing with this log storage system is reinventing sqlite, because systemd has a fairly new and not battle-tested database implementation, it has shortcomings. I would argue that either they use plain text logs like before or use sqlite but they should not reinvent a new db.
- pmlnr 7y agoSystemd is the very manifestation of NIH and reinvention, they are incapable of using existing tech.
- sjwright 7y agoGo back to the horse and cart then. Our fancy horseless carriage is just a reinvention of existing tech.
- adwn 7y ago> Sqlite is battle-tested though, it has one of the most comprehensive test suite and is used on billions of devices already. That's great, but you've completely ignored theamk's argument that sqlite is the wrong tool for the job.
- realusername 7y agoI don't agree with that argument, sqlite is totally the right tool for the job, it provides consistency, reliability, an api you can query anything upon, transactions, indexing, tools... And fits into files. You currently can't trust systemd log storage at all, even on my own laptop it manages to have some corruption... Querying is very poor and inefficient, indexing is unreliable... And that's exactly what I expect, building a database is hard.
- growse 7y agoSQLite doesn't work reliably on NFS, which rules it out for any PXE-booted system with an NFS root.
- SQLite 7y agoThe issue with NFS and SQLite is that posix advisory locks do not work, or do not work well, on many NFS installations. As long as advisory locks work over NFS, or as long as there is only one client trying to access the database at a time, so that locks are not really needed, SQLite works fine over NFS. NFS is a lot slower, but that's just the nature of NFS and SQLite can't do anything about that. You can ask SQLite to use dot-file locking instead of posix advisory locking. Or you can ask it to simply ignore locking all together. In both cases, SQLite will work on even a broken NFS system, though with the corresponding concurrency limitations.
- growse 7y agoFair. In my experience, the fact that NFS is both pervasive and often broken, for something like a system-wide log manager you can't always assume that your root fs is reliable. Is dot-file locking a compile- or run-time option?
- rurban 7y agosqlite is a dirty hack. just because people like hacks doesn't mean it's ready for the serious stuff. it is not. https://research.checkpoint.com/2019/select-code_execution-from-using-sqlite/ https://research.checkpoint.com/2019/select-code_execution-f...
- realusername 7y agoIt's certainly more ready than a custom hacked-up database which has data corruption even on my own laptop.
- saati 7y agoNo local logging will work without disk space, and if the disk is bad anything can get corrupted.
- mantap 7y agoIf you are low on disk space you want something predictable to happen: the oldest logs are deleted and the newest logs are written. This would be a nightmare to achieve with SQLite, as said above it's just way too complicated.
- saati 7y agoIf you actually care about your logs you should log to a centralized service anyway, and if any of your computers actually ran out of disk space you should fire your sysadmins/sres for incompetence.
- dijit 7y agoThis comment denotes, strongly, the dissonance that occurs when talking about something so general. It’s my opinion (as an infrastructure type) that centralised binary logging is an absolute necessity above 20 instances. But that leaves out every person with a laptop, or those people with a very low volume of little instances. Structured logging is difficult, that’s why logstash is so flexible about input and conversions. And I think that was the impetus for systemd’s journals; to log structurally. However. The parent is wrong if he thinks decisions should be made under low disk conditions. Having a rotate rule to keep your sliding window of logs to a certain size is fine. But your servers (and laptops, and desktops) should not be taking arbitrary decisions based on this kind of condition.
- secabeen 7y ago> This comment denotes, strongly, the dissonance that occurs when talking about something so general. > It’s my opinion (as an infrastructure type) that centralised binary logging is an absolute necessity above 20 instances. > But that leaves out every person with a laptop, or those people with a very low volume of little instances. Yep, it also leaves out people on embedded hardware, and on devices that are in remote locations without access to the Internet, or in a car, or on metered Internet, etc. It's hard to hold in your head just how broadly Linux is used these days.