5 ms·

Sqlite is battle-tested though, it has one of the most comprehensive test suite and is used on billions of devices already. What systemd is doing with this log

Sqlite is battle-tested though, it has one of the most comprehensive test suite and is used on billions of devices already. What systemd is doing with this log storage system is reinventing sqlite, because systemd has a fairly new and not battle-tested database implementation, it has shortcomings. I would argue that either they use plain text logs like before or use sqlite but they should not reinvent a new db.

pmlnr 7y ago
Systemd is the very manifestation of NIH and reinvention, they are incapable of using existing tech.
- sjwright 7y ago
  Go back to the horse and cart then. Our fancy horseless carriage is just a reinvention of existing tech.
adwn 7y ago
> Sqlite is battle-tested though, it has one of the most comprehensive test suite and is used on billions of devices already. That's great, but you've completely ignored theamk's argument that sqlite is the wrong tool for the job.
- realusername 7y ago
  I don't agree with that argument, sqlite is totally the right tool for the job, it provides consistency, reliability, an api you can query anything upon, transactions, indexing, tools... And fits into files. You currently can't trust systemd log storage at all, even on my own laptop it manages to have some corruption... Querying is very poor and inefficient, indexing is unreliable... And that's exactly what I expect, building a database is hard.
  - growse 7y ago
    SQLite doesn't work reliably on NFS, which rules it out for any PXE-booted system with an NFS root.
    - SQLite 7y ago
      The issue with NFS and SQLite is that posix advisory locks do not work, or do not work well, on many NFS installations. As long as advisory locks work over NFS, or as long as there is only one client trying to access the database at a time, so that locks are not really needed, SQLite works fine over NFS. NFS is a lot slower, but that's just the nature of NFS and SQLite can't do anything about that. You can ask SQLite to use dot-file locking instead of posix advisory locking. Or you can ask it to simply ignore locking all together. In both cases, SQLite will work on even a broken NFS system, though with the corresponding concurrency limitations.
      - growse 7y ago
        Fair. In my experience, the fact that NFS is both pervasive and often broken, for something like a system-wide log manager you can't always assume that your root fs is reliable. Is dot-file locking a compile- or run-time option?
rurban 7y ago
sqlite is a dirty hack. just because people like hacks doesn't mean it's ready for the serious stuff. it is not. https://research.checkpoint.com/2019/select-code_execution-from-using-sqlite/ https://research.checkpoint.com/2019/select-code_execution-f...
- realusername 7y ago
  It's certainly more ready than a custom hacked-up database which has data corruption even on my own laptop.

pmlnr 7y ago

Systemd is the very manifestation of NIH and reinvention, they are incapable of using existing tech.

sjwright 7y ago
Go back to the horse and cart then. Our fancy horseless carriage is just a reinvention of existing tech.

adwn 7y ago

> Sqlite is battle-tested though, it has one of the most comprehensive test suite and is used on billions of devices already. That's great, but you've completely ignored theamk's argument that sqlite is the wrong tool for the job.

realusername 7y ago
I don't agree with that argument, sqlite is totally the right tool for the job, it provides consistency, reliability, an api you can query anything upon, transactions, indexing, tools... And fits into files. You currently can't trust systemd log storage at all, even on my own laptop it manages to have some corruption... Querying is very poor and inefficient, indexing is unreliable... And that's exactly what I expect, building a database is hard.
- growse 7y ago
  SQLite doesn't work reliably on NFS, which rules it out for any PXE-booted system with an NFS root.
  - SQLite 7y ago
    The issue with NFS and SQLite is that posix advisory locks do not work, or do not work well, on many NFS installations. As long as advisory locks work over NFS, or as long as there is only one client trying to access the database at a time, so that locks are not really needed, SQLite works fine over NFS. NFS is a lot slower, but that's just the nature of NFS and SQLite can't do anything about that. You can ask SQLite to use dot-file locking instead of posix advisory locking. Or you can ask it to simply ignore locking all together. In both cases, SQLite will work on even a broken NFS system, though with the corresponding concurrency limitations.
    - growse 7y ago
      Fair. In my experience, the fact that NFS is both pervasive and often broken, for something like a system-wide log manager you can't always assume that your root fs is reliable. Is dot-file locking a compile- or run-time option?

rurban 7y ago

sqlite is a dirty hack. just because people like hacks doesn't mean it's ready for the serious stuff. it is not. https://research.checkpoint.com/2019/select-code_execution-from-using-sqlite/ https://research.checkpoint.com/2019/select-code_execution-f...

realusername 7y ago
It's certainly more ready than a custom hacked-up database which has data corruption even on my own laptop.