3 ms·

> OTOH "systemctl start httpd.service" I've also had issues with My/MariaDB not starting and the output capture by systemd and journald be absolutely useless i

> OTOH "systemctl start httpd.service" I've also had issues with My/MariaDB not starting and the output capture by systemd and journald be absolutely useless in debugging the issue (i.e., nothing shows up). Generally resort to running mysqld manually at the CLI to either get some kind of stdout/err output or maybe sent to the error log file. Of course if there's an error really early in the program/daemon initialization you used to be able to run strace against the start-up script, but that's not possible with systemctl. I've run into that situation when (e.g.) /var/lib/mysql was a link to a mount point (secondary drive) and SELinux or AppArmour kicked in.

ilyt 3y ago
> Of course if there's an error really early in the program/daemon initialization you used to be able to run strace against the start-up script, but that's not possible with systemctl. I mean it's PITA but you can just add strace command to start of ExecStart... > I've run into that situation when (e.g.) /var/lib/mysql was a link to a mount point (secondary drive) and SELinux or AppArmour kicked in. Yeah we hit that with some units that had maintainers preemptively setting security policy so distro-blessed paths were only writable ones for a given unit. But on plus side all could be fixed in few lines of overrides vs putting whole init script into the repo > I've also had issues with My/MariaDB not starting and the output capture by systemd and journald be absolutely useless in debugging the issue (i.e., nothing shows up). Generally resort to running mysqld manually at the CLI to either get some kind of stdout/err output or maybe sent to the error log file. Do you have systemd unit for that mysql problem ? There is option to redirect stdout/err to logs directly but systemd is too stupid to extract that logs back and show it in the status command. I definitely saw that issue you mentioned (it almost looked like it wasn't logged anywhere if app exited too quickly). But I don't see it on recent debians at the very least. ● mariadb.service - MariaDB 10.5.15 database server Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2023-05-08 12:41:26 CEST; 1 months 21 days ago Docs: man:mariadbd(8) https://mariadb.com/kb/en/library/systemd/ Process: 2913787 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS) Process: 2913788 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) Process: 2913790 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`cd /usr/bin/..; /usr/bin/galera_recovery`; [ $? -eq 0 ] && systemct> Process: 2913887 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) Process: 2913889 ExecStartPost=/etc/mysql/debian-start (code=exited, status=0/SUCCESS) Main PID: 2913870 (mariadbd) Status: "Taking your SQL requests now..." Tasks: 18 (limit: 2340) Memory: 303.5M CPU: 10h 32min 54.873s CGroup: /system.slice/mariadb.service └─2913870 /usr/sbin/mariadbd May 08 12:41:26 hostname systemd[1]: Starting MariaDB 10.5.15 database server... May 08 12:41:26 hostname mariadbd[2913870]: 2023-05-08 12:41:26 0 [Note] /usr/sbin/mariadbd (mysqld 10.5.15-MariaDB-0+deb11u1) starting as process 2913870 ... May 08 12:41:26 hostname systemd[1]: Started MariaDB 10.5.15 database server. May 08 12:41:26 hostname /etc/mysql/debian-start[2913894]: Looking for 'mysql' as: /usr/bin/mysql May 08 12:41:26 hostname /etc/mysql/debian-start[2913894]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck May 08 12:41:26 hostname /etc/mysql/debian-start[2913894]: This installation of MariaDB is already upgraded to 10.5.15-MariaDB. May 08 12:41:26 hostname /etc/mysql/debian-start[2913894]: There is no need to run mysql_upgrade again for 10.5.15-MariaDB. May 08 12:41:26 hostname /etc/mysql/debian-start[2913894]: You can use --force if you still want to run mysql_upgrade
- throw0101a 3y ago
  > Do you have systemd unit for that mysql problem ? Yeah: whatever ships/shipped with Debian. (Which we used at my last job, where I had this issue more than once. Don't run the same stack at my new job.)

ilyt 3y ago

> Of course if there's an error really early in the program/daemon initialization you used to be able to run strace against the start-up script, but that's not possible with systemctl. I mean it's PITA but you can just add strace command to start of ExecStart... > I've run into that situation when (e.g.) /var/lib/mysql was a link to a mount point (secondary drive) and SELinux or AppArmour kicked in. Yeah we hit that with some units that had maintainers preemptively setting security policy so distro-blessed paths were only writable ones for a given unit. But on plus side all could be fixed in few lines of overrides vs putting whole init script into the repo > I've also had issues with My/MariaDB not starting and the output capture by systemd and journald be absolutely useless in debugging the issue (i.e., nothing shows up). Generally resort to running mysqld manually at the CLI to either get some kind of stdout/err output or maybe sent to the error log file. Do you have systemd unit for that mysql problem ? There is option to redirect stdout/err to logs directly but systemd is too stupid to extract that logs back and show it in the status command. I definitely saw that issue you mentioned (it almost looked like it wasn't logged anywhere if app exited too quickly). But I don't see it on recent debians at the very least. ● mariadb.service - MariaDB 10.5.15 database server Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2023-05-08 12:41:26 CEST; 1 months 21 days ago Docs: man:mariadbd(8) https://mariadb.com/kb/en/library/systemd/ Process: 2913787 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS) Process: 2913788 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) Process: 2913790 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`cd /usr/bin/..; /usr/bin/galera_recovery`; [ $? -eq 0 ] && systemct> Process: 2913887 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS) Process: 2913889 ExecStartPost=/etc/mysql/debian-start (code=exited, status=0/SUCCESS) Main PID: 2913870 (mariadbd) Status: "Taking your SQL requests now..." Tasks: 18 (limit: 2340) Memory: 303.5M CPU: 10h 32min 54.873s CGroup: /system.slice/mariadb.service └─2913870 /usr/sbin/mariadbd May 08 12:41:26 hostname systemd[1]: Starting MariaDB 10.5.15 database server... May 08 12:41:26 hostname mariadbd[2913870]: 2023-05-08 12:41:26 0 [Note] /usr/sbin/mariadbd (mysqld 10.5.15-MariaDB-0+deb11u1) starting as process 2913870 ... May 08 12:41:26 hostname systemd[1]: Started MariaDB 10.5.15 database server. May 08 12:41:26 hostname /etc/mysql/debian-start[2913894]: Looking for 'mysql' as: /usr/bin/mysql May 08 12:41:26 hostname /etc/mysql/debian-start[2913894]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck May 08 12:41:26 hostname /etc/mysql/debian-start[2913894]: This installation of MariaDB is already upgraded to 10.5.15-MariaDB. May 08 12:41:26 hostname /etc/mysql/debian-start[2913894]: There is no need to run mysql_upgrade again for 10.5.15-MariaDB. May 08 12:41:26 hostname /etc/mysql/debian-start[2913894]: You can use --force if you still want to run mysql_upgrade

throw0101a 3y ago
> Do you have systemd unit for that mysql problem ? Yeah: whatever ships/shipped with Debian. (Which we used at my last job, where I had this issue more than once. Don't run the same stack at my new job.)