7 ms·

These problems are all solved without destroying the Unix philosophy completely: starting services - daemontools. journaling - daemontools billion different

by d0 13y ago

These problems are all solved without destroying the Unix philosophy completely: starting services - daemontools. journaling - daemontools billion different sentinel/watch type apps - daemontools. dependencies - just don't do it. This is the old age of power sequencers again. It's coupling. Daemontools is a set of tiny programs that talk to each other, are fully privilege separated and guarantee reliability. Why they can't do this for systemd, I don't know. If you remove the DJB path conventions from it (the major objections), it's the right tool for the job.

tc104 13y ago
Oh don't start babbling about Unix Philosophy. It's wearing a little thin. Even the people who actually worked on Unix are sick of hearing this appeal-to-authority, "BECAUSE UNIX PHILOSOPHY" nonsense by now.
FooBarWidget 13y ago
> starting services - daemontools Daemontools is great, but please don't pretend it's a proper general-purpose init system. How do you start a certain service before another one? I suppose you can hack that inside the run script but then you're yet again duplicating logic, badly. > journaling - daemontools What? Daemontools provides a nice utility for logging a process stdout/stderr to a file. That's it. That doesn't come close to the problem that journald tries to solve. > billion different sentinel/watch type apps - daemontools. This is about the only thing that daemontools solves. > dependencies - just don't do it. This is the old age of power sequencers again. It's coupling. And here comes the mythical "coupling is bad" statement while completely ignoring the problem. No, I don't want my NFS daemon to be started before my network connection is up, thank you very much. Your post shows a complete lack of understanding of the problem.
- d0 13y ago
  > Daemontools is great, but please don't pretend it's a proper general-purpose init system. How do you start a certain service before another one? I suppose you can hack that inside the run script but then you're yet again duplicating logic, badly. I'm not saying its a replacement. The model is suitable as a replacement i.e. inspiration should be taken from it. What? Daemontools provides a nice utility for logging a process stdout/stderr to a file. That's it. That doesn't come close to the problem that journald tries to solve. It guarantees that the logs are stamped externally, are committed to disk and that the logs haven't been tampered with by isolating them from other processes and user accounts that can modify or write to them. Don't need journaling on top, just separation of concerns. Indexing - don't need it. No one does. That's an external problem solved by syslog collectors like Splunk. As someone who deals with up to 500Gb of logs a day, I know my shit. Systemd doesn't solve a thing here. In fact it adds overhead to a solved problem. And here comes the mythical "coupling is bad" statement while completely ignoring the problem. No, I don't want my NFS daemon to be started before my network connection is up, thank you very much. Well what the hell does your NFS daemon do when your network goes down, your adapter gets hotplugged after a failure or someone falls over the cable? There is so little of the problem solved by systemd it's unbelievable. The RIGHT solution is to make the processes resilient to failure conditions like this.
  - edwintorok 13y ago
    > Indexing - don't need it. No one does. That's an external problem solved by syslog collectors like Splunk. As someone who deals with up to 500Gb of logs a day, I know my shit. Systemd doesn't solve a thing here. In fact it adds overhead to a solved problem. Does systemd's journal still use a binary format? That is a no-go for me. Inevitably it will get corrupted, and then you rely on the journal reader being able to recover your logs, or fix bugs in it until it does. I'd be more willing to consider systemd as an alternative if it kept using a human-readable log format, that can be read with cat/tail in a worst-case scenario. If it wants to index it, it can keep an index on the side, no big deal if that gets corrupted/out of date, as it can always be rebuilt.
    - FooBarWidget 13y ago
      I don't buy this argument. SQLite and PostgreSQL's databases are binary too, but that's ok?
      - edwintorok 13y ago
        A database can afford to fsync to ensure consistency, and they've been heavily tested to ensure it works properly. And even then its still possible that the file becomes corrupt, if the OS, or the disk lies about fsync: https://www.sqlite.org/howtocorrupt.html https://www.sqlite.org/howtocorrupt.html Has journald been tested on how well it copes with sudden reboots, kernel panics, powerloss, etc.? When something goes wrong you usually want to be able to still read your logs to figure out what happened, and you may not even be able to boot the system properly.
        reidrac 13y ago
        > Has journald been tested on how well it copes with sudden reboots, kernel panics, powerloss, etc.? Are you assuming it hasn't been tested in these situations just because you don't know?
        13y ago
nhaehnle 13y ago
Dependencies between services exist, and for good reason (e.g. various services depending on a database). How do you propose to deal with them?
- d0 13y ago
  It's pretty basic software design. A few solutions: 1. Web servers return error pages when databases are unavailable. They don't go down. 2. Postfix doesn't die if your virtual maps database server is not available. It waits patiently. This isn't rocket science.
  - nailer 13y ago
    I'd rather Postfix fail early, than wait until $TIMEOUT to discover that a maps source isn't available.
    - mercurial 13y ago
      I think what parent is trying to say in so many words is "I have no use case for tracking service dependencies, therefore it is pointless and should be cut out". An unfortunately common attitude among technical people.
      - d0 13y ago
        No. I'm saying that relying on startup sequencing is a bad idea full stop.
        the_mitsuhiko 13y ago
        That's why systemd does dependency tracking and socket activation instead of sequencing.
        d0 13y ago
        It needs to do neither of those as well.
Fasebook 13y ago
> This is the old age of power sequencers again. It's coupling. Is this referring to power transmission or something?
- d0 13y ago
  No. Back in the bad old days of big Unix machines and mainframes we used power sequencers to bring physical bits of hardware up in order. This entire problem has now moved into the software space. With hardware, dependencies were painful, expensive and relied on lots of voodoo and dependency resolution. The same with software. Eventually the hardware manufacturers and the OS people got together and threw this out of the window. Now Linux is about to fail to learn from those mistakes like it failed to learn about audio subsystems, firewalls, network configuration, desktop environments, graphics architecture etc.
SEJeff 13y ago
Please suggest something a bit better. Daemon tools doesn't even support limits. As a result, I had to rip it out of a ~2k node cluster if favor of monit.
- d0 13y ago
  I'm not suggesting daemontools. I'm suggesting "not systemd" and "something that includes the valuable lessons that daemontools taught us". The first of which is not to fuck up the Unix philosophy by producing a monolithic pile of junk.
- mydecombinator 13y ago
  I'm not sure which limits you are talking about. These, or others? > http://cr.yp.to/daemontools/softlimit.html http://cr.yp.to/daemontools/softlimit.html
nailer 13y ago
This is a little off topic, but DJB is obviously incredibly talented, but frequently out of step (rightly or wrong) with the general Linux community. I think it would be excellent if DJB had his own OS - maybe with a Linux kernel, but with a completely different userspace like Android. Specifically: - the init - the packaging system and file heirarchy - the daemons I'd love to see and use it.
mydecombinator 13y ago
You can also use 'runit', which has quite similar design, but comes pre-packaged for a lot of systems, and can run as init, too (only if you want to).

by d0 13y ago

tc104 13y ago

Oh don't start babbling about Unix Philosophy. It's wearing a little thin. Even the people who actually worked on Unix are sick of hearing this appeal-to-authority, "BECAUSE UNIX PHILOSOPHY" nonsense by now.

FooBarWidget 13y ago

> starting services - daemontools Daemontools is great, but please don't pretend it's a proper general-purpose init system. How do you start a certain service before another one? I suppose you can hack that inside the run script but then you're yet again duplicating logic, badly. > journaling - daemontools What? Daemontools provides a nice utility for logging a process stdout/stderr to a file. That's it. That doesn't come close to the problem that journald tries to solve. > billion different sentinel/watch type apps - daemontools. This is about the only thing that daemontools solves. > dependencies - just don't do it. This is the old age of power sequencers again. It's coupling. And here comes the mythical "coupling is bad" statement while completely ignoring the problem. No, I don't want my NFS daemon to be started before my network connection is up, thank you very much. Your post shows a complete lack of understanding of the problem.

d0 13y ago
> Daemontools is great, but please don't pretend it's a proper general-purpose init system. How do you start a certain service before another one? I suppose you can hack that inside the run script but then you're yet again duplicating logic, badly. I'm not saying its a replacement. The model is suitable as a replacement i.e. inspiration should be taken from it. What? Daemontools provides a nice utility for logging a process stdout/stderr to a file. That's it. That doesn't come close to the problem that journald tries to solve. It guarantees that the logs are stamped externally, are committed to disk and that the logs haven't been tampered with by isolating them from other processes and user accounts that can modify or write to them. Don't need journaling on top, just separation of concerns. Indexing - don't need it. No one does. That's an external problem solved by syslog collectors like Splunk. As someone who deals with up to 500Gb of logs a day, I know my shit. Systemd doesn't solve a thing here. In fact it adds overhead to a solved problem. And here comes the mythical "coupling is bad" statement while completely ignoring the problem. No, I don't want my NFS daemon to be started before my network connection is up, thank you very much. Well what the hell does your NFS daemon do when your network goes down, your adapter gets hotplugged after a failure or someone falls over the cable? There is so little of the problem solved by systemd it's unbelievable. The RIGHT solution is to make the processes resilient to failure conditions like this.
- edwintorok 13y ago
  > Indexing - don't need it. No one does. That's an external problem solved by syslog collectors like Splunk. As someone who deals with up to 500Gb of logs a day, I know my shit. Systemd doesn't solve a thing here. In fact it adds overhead to a solved problem. Does systemd's journal still use a binary format? That is a no-go for me. Inevitably it will get corrupted, and then you rely on the journal reader being able to recover your logs, or fix bugs in it until it does. I'd be more willing to consider systemd as an alternative if it kept using a human-readable log format, that can be read with cat/tail in a worst-case scenario. If it wants to index it, it can keep an index on the side, no big deal if that gets corrupted/out of date, as it can always be rebuilt.
  - FooBarWidget 13y ago
    I don't buy this argument. SQLite and PostgreSQL's databases are binary too, but that's ok?
    - edwintorok 13y ago
      A database can afford to fsync to ensure consistency, and they've been heavily tested to ensure it works properly. And even then its still possible that the file becomes corrupt, if the OS, or the disk lies about fsync: https://www.sqlite.org/howtocorrupt.html https://www.sqlite.org/howtocorrupt.html Has journald been tested on how well it copes with sudden reboots, kernel panics, powerloss, etc.? When something goes wrong you usually want to be able to still read your logs to figure out what happened, and you may not even be able to boot the system properly.
      - reidrac 13y ago
        > Has journald been tested on how well it copes with sudden reboots, kernel panics, powerloss, etc.? Are you assuming it hasn't been tested in these situations just because you don't know?
        13y ago

nhaehnle 13y ago

Dependencies between services exist, and for good reason (e.g. various services depending on a database). How do you propose to deal with them?

d0 13y ago
It's pretty basic software design. A few solutions: 1. Web servers return error pages when databases are unavailable. They don't go down. 2. Postfix doesn't die if your virtual maps database server is not available. It waits patiently. This isn't rocket science.
- nailer 13y ago
  I'd rather Postfix fail early, than wait until $TIMEOUT to discover that a maps source isn't available.
  - mercurial 13y ago
    I think what parent is trying to say in so many words is "I have no use case for tracking service dependencies, therefore it is pointless and should be cut out". An unfortunately common attitude among technical people.
    - d0 13y ago
      No. I'm saying that relying on startup sequencing is a bad idea full stop.
      - the_mitsuhiko 13y ago
        That's why systemd does dependency tracking and socket activation instead of sequencing.
        d0 13y ago
        It needs to do neither of those as well.

Fasebook 13y ago

> This is the old age of power sequencers again. It's coupling. Is this referring to power transmission or something?

d0 13y ago
No. Back in the bad old days of big Unix machines and mainframes we used power sequencers to bring physical bits of hardware up in order. This entire problem has now moved into the software space. With hardware, dependencies were painful, expensive and relied on lots of voodoo and dependency resolution. The same with software. Eventually the hardware manufacturers and the OS people got together and threw this out of the window. Now Linux is about to fail to learn from those mistakes like it failed to learn about audio subsystems, firewalls, network configuration, desktop environments, graphics architecture etc.

SEJeff 13y ago

Please suggest something a bit better. Daemon tools doesn't even support limits. As a result, I had to rip it out of a ~2k node cluster if favor of monit.

d0 13y ago
I'm not suggesting daemontools. I'm suggesting "not systemd" and "something that includes the valuable lessons that daemontools taught us". The first of which is not to fuck up the Unix philosophy by producing a monolithic pile of junk.
mydecombinator 13y ago
I'm not sure which limits you are talking about. These, or others? > http://cr.yp.to/daemontools/softlimit.html http://cr.yp.to/daemontools/softlimit.html

nailer 13y ago

This is a little off topic, but DJB is obviously incredibly talented, but frequently out of step (rightly or wrong) with the general Linux community. I think it would be excellent if DJB had his own OS - maybe with a Linux kernel, but with a completely different userspace like Android. Specifically: - the init - the packaging system and file heirarchy - the daemons I'd love to see and use it.

mydecombinator 13y ago

You can also use 'runit', which has quite similar design, but comes pre-packaged for a lot of systems, and can run as init, too (only if you want to).