3 ms·

What servers usually run as root? Some may start as root, but usually drop privileges for the actual server processes quickly, eg. apache, nginx, sshd. Nothing

What servers usually run as root? Some may start as root, but usually drop privileges for the actual server processes quickly, eg. apache, nginx, sshd. Nothing that actually does the "serving" or accesses data should be running as root.

GekkePrutser 6y ago
No but the logfile writers are usually running as root AFAIK. And this is what tends to fill up the disk.
- edoceo 6y ago
  Mine don't run as root.
  - vntok 6y ago
    You should fix it then, takes no time at all.
    - repiret 6y ago
      Thats not really something that needs fixing.
- derefr 6y ago
  On systemd systems, logfiles are written to disk under the journald user, `systemd-journal`.
  - deleted 6y ago
    [deleted]
  - kiwijamo 6y ago
    Is that true for all logfiles? I still have plenty of daemons (by default) writing directly to some file in /var/log eg EXIM, Apache, and the like. Also plenty of system stuff still write to files in that directory. And yes this is a machine that uses systemd.
    - comex 6y ago
      But those daemons don’t usually have their own log writer processes running as root, do they? Instead, either the log file is accessible by the user the daemon is running as, or the daemon opens the log file as root before dropping privileges for the rest of its operation.
    - stonesweep 6y ago
      Most vendors (Debian/Ubuntu, RHEL/clones, etc.) add a hook into rsyslog to be a partner with the systemd logger and write out text files next to the journal - they realize that a lot of people dislike dealing with journalctl (I'm one of them) and provide an alternate hook already installed and working for you behind the scenes. This is for daemons using syslog methodology, not direct writers like apache/nginx/mysql/etc; think more like cron, systemd, chrony, NetworkManager, and so forth. The vendors are not all aligned on what goes where (example: on RHEL, pacemaker/crm write to their own logs buy on openSUSE they're sent to syslog) - the actual results differ slightly from vendor to vendor. DIY distros like Arch do not implement the rsyslog backend by default, you have to set it up yourself following the wiki - only journalctl is there by default.
  - GekkePrutser 6y ago
    Ah good point, I use Alpine on all my servers so it's more traditional logs.
- chousuke 6y ago
  My rule of thumb to avoid these issues is that application/server data gets its own dedicated volume that contains nothing else: logs get their own volume, and root its own. It's an especially bad idea for an application to put its data and logs in the same directory where its binaries reside. That way, even if your log volume or root somehow fills up before monitoring had a chance to react, your service is unaffected. You can even catch issues pre-emptively by keeping log volumes small so that weird behaviour is likely to trigger an alert before anything goes truly wrong. On cloud instances, it's silly to put anything on the instance root volume (on AWS, I keep them at the default 8 GB; it's never been a problem) when you can just attach an arbitrary number of additional disks. Container systems would use persistent volumes, and with physical servers, you use LVM or equivalent. This solves most disk allocation issues and makes operations easy when you need more space.
znpy 6y ago
It used to be common, before "the cloud", to have many apparently unnecessary partitions in a server install. One for /, one for /var, one for /home, one for swap at the low sector numbers... The idea is that /var filling up would not make the system unrecoverable.

GekkePrutser 6y ago

No but the logfile writers are usually running as root AFAIK. And this is what tends to fill up the disk.

edoceo 6y ago
Mine don't run as root.
- vntok 6y ago
  You should fix it then, takes no time at all.
  - repiret 6y ago
    Thats not really something that needs fixing.
derefr 6y ago
On systemd systems, logfiles are written to disk under the journald user, `systemd-journal`.
- deleted 6y ago
  [deleted]
- kiwijamo 6y ago
  Is that true for all logfiles? I still have plenty of daemons (by default) writing directly to some file in /var/log eg EXIM, Apache, and the like. Also plenty of system stuff still write to files in that directory. And yes this is a machine that uses systemd.
  - comex 6y ago
    But those daemons don’t usually have their own log writer processes running as root, do they? Instead, either the log file is accessible by the user the daemon is running as, or the daemon opens the log file as root before dropping privileges for the rest of its operation.
  - stonesweep 6y ago
    Most vendors (Debian/Ubuntu, RHEL/clones, etc.) add a hook into rsyslog to be a partner with the systemd logger and write out text files next to the journal - they realize that a lot of people dislike dealing with journalctl (I'm one of them) and provide an alternate hook already installed and working for you behind the scenes. This is for daemons using syslog methodology, not direct writers like apache/nginx/mysql/etc; think more like cron, systemd, chrony, NetworkManager, and so forth. The vendors are not all aligned on what goes where (example: on RHEL, pacemaker/crm write to their own logs buy on openSUSE they're sent to syslog) - the actual results differ slightly from vendor to vendor. DIY distros like Arch do not implement the rsyslog backend by default, you have to set it up yourself following the wiki - only journalctl is there by default.
- GekkePrutser 6y ago
  Ah good point, I use Alpine on all my servers so it's more traditional logs.
chousuke 6y ago
My rule of thumb to avoid these issues is that application/server data gets its own dedicated volume that contains nothing else: logs get their own volume, and root its own. It's an especially bad idea for an application to put its data and logs in the same directory where its binaries reside. That way, even if your log volume or root somehow fills up before monitoring had a chance to react, your service is unaffected. You can even catch issues pre-emptively by keeping log volumes small so that weird behaviour is likely to trigger an alert before anything goes truly wrong. On cloud instances, it's silly to put anything on the instance root volume (on AWS, I keep them at the default 8 GB; it's never been a problem) when you can just attach an arbitrary number of additional disks. Container systems would use persistent volumes, and with physical servers, you use LVM or equivalent. This solves most disk allocation issues and makes operations easy when you need more space.

znpy 6y ago

It used to be common, before "the cloud", to have many apparently unnecessary partitions in a server install. One for /, one for /var, one for /home, one for swap at the low sector numbers... The idea is that /var filling up would not make the system unrecoverable.