Pages

Thursday, 3 October 2019

Bad experience on systemd-tmpfiles-setup.service

KeyWords:  /run/httpd, /run/nologin

After rebooting CentOS 7.7, Apache httpd was not able to start up.

[root@test2 run]# systemctl start httpd
Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.

Check /var/log/httpd/error.log

[Thu Oct 03 17:47:29.360416 2019] [auth_digest:error] [pid 129175] (2)No such file or directory: AH01762: Failed to create shared memory segment on file /run/httpd/authdigest_shm.129175

The reason is obvious. After checking I found /run/httpd didn't exist at all. So who should have created /run/httpd. The answer is

systemd-tmpfiles-setup.service

This service reads config files under /usr/lib/tmpfiles.d/.
Apache package contains a file under it.

[root@test2 tmpfiles.d]# rpm -ql httpd | grep tmpfiles
/usr/lib/tmpfiles.d/httpd.conf

cat it:

d /run/httpd   710 root apache
d /run/httpd/htcacheclean   700 apache apache

This service is a static service.

[root@test2 tmpfiles.d]# cat /usr/lib/systemd/system/systemd-tmpfiles-setup.service
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=Create Volatile Files and Directories
Documentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8)
DefaultDependencies=no
Conflicts=shutdown.target
After=systemd-readahead-collect.service systemd-readahead-replay.service local-fs.target systemd-sysusers.service
Before=sysinit.target shutdown.target
RefuseManualStop=yes

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/systemd-tmpfiles --create --remove --boot --exclude-prefix=/dev

Normally it runs automatically when OS starts up. But for this time, it didn't. So the required /run/httpd was not created.

When I checked its status, it was dead. Then I started it. and /run/httpd was created and httpd was able to start.

After it's started another issue came that normal users cannot log in.

System is booting up. See pam_nologin(8)
Authentication failed.
This is because /var/run/nologin file was created after systemd-tmpfiles-setup.service was started. Once this file exists, normal account cannot log in.

After remove /run/nologin manually, normal user was able to log in the system.


Actually, besides httpd, there are many apps depending on this service.

[root@test2 tmpfiles.d]# ls /usr/lib/tmpfiles.d/
dovecot.conf      named.conf     selinux-policy.conf
etc.conf          openldap.conf  sudo.conf
httpd.conf        pam.conf       svnserve.conf
initscripts.conf  python.conf    systemd.conf
iscsi.conf        rpcbind.conf   systemd-nologin.conf
legacy.conf       rpm.conf       tmp.conf
libselinux.conf   samba.conf     tuned.conf
lvm2.conf         sap.conf       var.conf
mariadb.conf      screen.conf    x11.conf

So this is a critical service that should run properly when the OS starts. If not, many apps will be affected. 

The best solution is to reboot the OS to make sure this service works well.



No comments:

Post a Comment