Startup and Shutdown
Overview
After a Linux installation the system has a configuration. This configuration consists of things like what filesystems to mount, what daemons to start, what mode ( single user or multi user mode ) to start and so on. How can we change this configuration ? What steps should we take to debug an issue at startup ? How should a system be shutdown ? What changes can be made to perform custom tasks during shutdown ? This chapter covers these topics.Any Linux system will follow the below process for starting up.
- The computer is powered on and the BIOS starts up. BIOS is built into the motherboard
and stands for Basic Input Output System. It will check the hardware such as keyboard, hard drive
and so on. Once that passes it will check the boot order. This is a BIOS setting
that we can change. Different computers have different methods of entering the BIOS.
Usually it invovles powering on and pressing one of the "F" keys. The boot order can
have devices such as hard drive, flash drive, cd rom. BIOS will check if the device is
bootable. If we installed a Linux system and then BIOS will check the Master Boot Record but
it can also happen that we have a USB that we want to use to install Linux. It will also
be bootable and have a MBR. MBR has been discussed before in the file system
section:
https://fog.ccsf.edu/~amittal/cs260a/fs/fs1.html
The BIOS will locate the loader in the MBR and give the control to it. In Linux the loader is usually the GRUB loader for Linux. If there are multiple operating systems located on the device on different partitions then the loader will present options to the user to select a particular one. The loader will then start a program ( the kernel ). - The loader gives control to the kernel. The kernel starts looking at the hardware
devices and loading the device drivers. If it finds a problem then it aborts with a panic
message. The kernel will display messages on the screen showing what it's doing. If it runs
into an issue it might allow a root login for us to fix the problem, restart the system
and try to run again.
The messages at this stage appear on the screen and can also be viewed later with the dmesg command. As an example; running the command on the hills server produces the following snippet. [amittal@hills ~]$ dmesg | grep "sda" [ 3.405746] sd 32:0:0:0: [sda] 314572800 512-byte logical blocks: (161 GB/150 GiB) [ 3.406519] sd 32:0:0:0: [sda] Write Protect is off [ 3.406524] sd 32:0:0:0: [sda] Mode Sense: 61 00 00 00 [ 3.406604] sd 32:0:0:0: [sda] Cache data unavailable [ 3.406607] sd 32:0:0:0: [sda] Assuming drive cache: write through [ 3.456552] sda: sda1 sda2 [ 3.460563] sd 32:0:0:0: [sda] Attached SCSI disk [ 9.825690] XFS (sda1): Mounting V5 Filesystem [ 10.066783] XFS (sda1): Ending clean mount [amittal@hills ~]$ The above shows that we have 2 partitions ( sda1, sda2 ) on the hard drive.
The kernel will create some processes that are not the conventional Linux process that we see with the "ps" command. These are used to start up the I/O subsystem, virtual memory , scheduler as an example. The kernel creates the first process called "init" which has a process of id 1. - This is where the differences in Linux systems start to appear. A distribution might have "SysVinit" or "Upstart" or "Systemd" as the init process with "SysVinit" being the oldest and "Systemd" being the newest. The old style init process is also called "init" creating some confusion. These are not the only init implementations. We have "OpenRC","launchD", "Runit". We shall only study "SysVinit" and "Systemd" in this section.
[amittal@hills ~]$ ps -aef | grep systemd | head root 1 0 0 Jan12 ? 00:30:46 /usr/lib/systemd/systemd --switched-root --system --deserialize 18 root Also the "init" program points to the systemd program [amittal@hills sbin]$ pwd /usr/sbin [amittal@hills sbin]$ ls -l init* lrwxrwxrwx. 1 root root 22 Jul 28 2021 init -> ../lib/systemd/systemd [amittal@hills sbin]$
The init process will have the process id of 1. The process id's are assigned sequentially and "1" is the first process id as zero is not used. The init process fathers all the rest of the processes directly or indirectly. A process is created from another process called the parent process. The init process keeps running until the system is shutdown. The init process can create other processes needed for startup and execute shell scripts also. The init process will also become parent to orphaned processes. We shall study processes in detail in a later chapter. We change the bootup configurations by either changing the arguments to the kernel or changing the configuration files. What configuration files do we change. This is dependent on the init implementation. An init process might use a concept called runlevel or something very similar.
Runlevel
A runlevel is an operating system state. It defines what services are currently running. The run level is a number from 0 to 6.Runlevel 0 shuts down the system Runlevel 1 single-user mode. No networking. Runlevel 2 multi-user mode without networking Runlevel 3 multi-user mode with networking and no graphics. Runlevel 4 user-definable Runlevel 5 multi-user mode with networking with graphics. Runlevel 6 reboots the system to restart it
When the system boots up only 1 runlevel is run. Runlevel 1 is single user mode. It is also called the maintenance mode. It allows us to fix issues like reset root password. We need physical access to the computer. We also have the "runlevel" command
[amittal@hills ~]$ runlevel N 3 [amittal@hills ~]$
As expected the hills server is running on level 3.
SysVinit
This is the old style implementation of init. The executable is at "/bin/init". It uses a configuration file called "/etc/inittab". We can find this file on the hills server but if we look at the contents we shall see that it is not being used because the hills server is using "SystemD". SysVinit uses the concept of runlevels. An entry in "inittab" is of the following format:
id:rstate:action:process
id -- A unique indentifier for the entry.
rstate -- Lists the runlevel to which the entry applies to
action -- How is the next field going to be run. Possible values
include: initdefault, sysinit, boot, bootwait, wait, and respawn.
If the value is "initdefault" then it identifies the default run level.
process -- Defines the process or script to execute.
Sample inittab file:
1 ap::sysinit:/sbin/autopush -f /etc/iu.ap
2 ap::sysinit:/sbin/soconfig -f /etc/sock2path
3 fs::sysinit:/sbin/rcS sysinit >/dev/msglog 2<>/dev/msglog /dev/msglog 2<>/dev/...
6 sS:s:wait:/sbin/rcS >/dev/msglog 2
7 s0:0:wait:/sbin/rc0 >/dev/msglog 2
8 s1:1:respawn:/sbin/rc1 >/dev/msglog 2
9 s2:23:wait:/sbin/rc2 >/dev/msglog 2
10 s3:3:wait:/sbin/rc3 >/dev/msglog 2
11 s5:5:wait:/sbin/rc5 >/dev/msglog 2
12 s6:6:wait:/sbin/rc6 >/dev/msglog 2
13 fw:0:wait:/sbin/uadmin 2 0 >/dev/msglog 2
14 of:5:wait:/sbin/uadmin 2 6 >/dev/msglog 2
15 rb:6:wait:/sbin/uadmin 2 1 >/dev/msglog 2
16 sc:234:respawn:/usr/lib/saf/sac -t 300
17 co:234:respawn:/usr/lib/saf/ttymon -g -h -p "`uname -n` console login: "
-T terminal-type -d /dev/console -l console
-m ldterm,ttcompat
From the above we notice the entry at line 4. This states that the default runlevel is 3. The man page for "inittab" contains a more detailed descripton of each of these fields. The "wait" entry in the theird field starts the process and waits till the process is up and running before moving to the next entry. Init starts the different scripts and processes. We can change the runlevel once we are logged in with the "telinit" command.
root@ajkumar08-PC:/home/ajay#
Prints the current runlevel of 5. "N" means there is no previous runlevel.
root@ajkumar08-PC:/home/ajay# runlevel
N 5
Change the runlevel to 3.
root@ajkumar08-PC:/home/ajay# telinit 3
Prints the previous run
root@ajkumar08-PC:/home/ajay# runlevel
5 3
root@ajkumar08-PC:/home/ajay# telinit 5
root@ajkumar08-PC:/home/ajay# runlevel
3 5
We can also use the "who -r" command to display the runlevel.
ajay@ajkumar08-PC:~$ who -r
run-level 5 2023-03-17 23:12 last=3
ajay@ajkumar08-PC:~$
The process now has to trigger scripts to start subsystems. It does this by running shell scripts depending on the runlevel we have. There is a folder for each runlevel. The below is a snapshot for the Debin 11 Linux folders.
See full image
Each folders contains scripts that are links to scripts in the "init.d" folder.
ajay@ajkumar08-PC:~$ cd /etc/rc0.d ajay@ajkumar08-PC:/etc/rc0.d$ ls -l total 0 lrwxrwxrwx 1 root root 20 Feb 12 19:45 K01alsa-utils -> ../init.d/alsa-utils lrwxrwxrwx 1 root root 22 Feb 12 19:48 K01avahi-daemon -> ../init.d/avahi-daemon lrwxrwxrwx 1 root root 19 Feb 12 19:45 K01bluetooth -> ../init.d/bluetooth lrwxrwxrwx 1 root root 22 Feb 12 19:49 K01cups-browsed -> ../init.d/cups-browsed lrwxrwxrwx 1 root root 14 Feb 12 19:51 K01gdm3 -> ../init.d/gdm3 lrwxrwxrwx 1 root root 20 Feb 12 19:01 K01hwclock.sh -> ../init.d/hwclock.sh lrwxrwxrwx 1 root root 23 Feb 19 17:12 K01lvm2-lvmpolld -> ../init.d/lvm2-lvmpolld lrwxrwxrwx 1 root root 20 Feb 12 19:02 K01networking -> ../init.d/networking lrwxrwxrwx 1 root root 18 Feb 12 19:46 K01plymouth -> ../init.d/plymouth lrwxrwxrwx 1 root root 37 Feb 12 19:49 K01pulseaudio-enable-autospawn -> ../init.d/pulseaudio-enable-autospawn lrwxrwxrwx 1 root root 17 Feb 12 19:02 K01rsyslog -> ../init.d/rsyslog lrwxrwxrwx 1 root root 15 Feb 12 19:49 K01saned -> ../init.d/saned lrwxrwxrwx 1 root root 27 Feb 12 19:48 K01speech-dispatcher -> ../init.d/speech-dispatcher lrwxrwxrwx 1 root root 14 Feb 12 19:02 K01udev -> ../init.d/udev lrwxrwxrwx 1 root root 29 Feb 12 19:48 K01unattended-upgrades -> ../init.d/unattended-upgrades
The scripts within each directory are named with either a capital S, or a capital K, followed by a two-digit number, followed by the name of the service being referenced. The files beginning with capital S represent scripts which are started upon entering that runlevel, while files beginning with capital K represent scripts which are stopped. The numbers specify the order in which the scripts should be executed. The main script "rc script" may reside on a folder "/etc/rc.d/rc" and is responsible for starting the scripts. It is possible to get the status of a subsystem and even start/stop these a subsystem manually. Hills service redirects the "service" command because it is running "SystemD".
To start the crond subsystem
service crond start
[amittal@hills /]$ service crond status
Redirecting to /bin/systemctl status crond.service
? crond.service - Command Scheduler
Loaded: loaded (/usr/lib/systemd/system/crond.service; enabled; vendor prese>
Active: active (running) since Thu 2023-01-12 09:57:48 PST; 2 months 4 days >
Main PID: 1629 (crond)
Tasks: 1 (limit: 75135)
Memory: 51.0M
CGroup: /system.slice/crond.service
+-1629 /usr/sbin/crond -n
To stop the crond subsystem
service crond stop
SystemD
SystemD came out in 2011 and was developed by engineers working for Red Hat. Some definitions related to the SystemD utility.Target
A target is a group of services. It is similar to a runlevel but is more generic.
Just like a default runlevel we have a default target.
Service
A service is a daemon that can be started and stopped. A daemon is a process
that runs in the background. We can think of a service as a susbsytem also. So
for example cron ( scheduler ) is a service.
Unit
A unit is a configuration file and can refer to a service, or a target or several other things.
The configuration files can be found in the following folders. /etc/systemd/system: Local configuration /run/systemd/system: Runtime configuration /lib/systemd/system: Distribution-wide configuration /usr/lib/systemd/system/: Contains default systemd unit configurations as per contained in the rpm.
SystemD will search the first folder for a unit configuration file and then the next one stopping till it finds one. This allows for customization. Suppose there is a distribution-wide configuration then we can place a custom unit configuration file of the same name in the lcoal configuration. A unit file can contain dependencies; these can be other units such as targets and services. A unit file begins with a section called unit.
File: graphical.target [Unit] Description=Graphical Interface Documentation=man:systemd.special(7) Requires=multi-user.target Wants=display-manager.service Conflicts=rescue.service rescue.target After=multi-user.target rescue.service rescue.target display-manager.service AllowIsolate=yes Unit dependencies are expressed though Requires, Wants, and Conflicts: Requires: A list of units that this unit depends on, which is started when this unit is started. Wants: A weaker form of Requires: the units listed are started but the current unit is not stopped if any of them fail. Conflicts: A negative dependency: the units listed are stopped when this one is started and, conversely, if one of them is started, this one is stopped Before: This unit should be started before the units listed After: This unit should be started after the units listed
A unit can also be a service and will have a service section.
File: [Unit] Description=OpenBSD Secure Shell server Documentation=man:sshd(8) man:sshd_config(5) After=network.target auditd.service ConditionPathExists=!/etc/ssh/sshd_not_to_be_run [Service] EnvironmentFile=-/etc/default/ssh ExecStartPre=/usr/sbin/sshd -t ExecStart=/usr/sbin/sshd -D $SSHD_OPTS ExecReload=/usr/sbin/sshd -t ExecReload=/bin/kill -HUP $MAINPID KillMode=process Restart=on-failure RestartPreventExitStatus=255 Type=notify RuntimeDirectory=sshd RuntimeDirectoryMode=0755 [Install] WantedBy=multi-user.target Alias=sshd.service
From the above:
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
ExecReload=/usr/sbin/sshd -t
The "ExecStart" as we can guess is for starting the service while "ExecReload" is for restarting it. The command "man systemd.service" has more information on these options. Targets have names ending in ".target". It is similar to a runlevel in that it is a state although the scheme is different and more flexible for selecton of services. We can create our own targets and use existing targets as dependencies.
The kernel will start the SystemD process by starting init. On the hills
server:
[amittal@hills sbin]$ pwd /sbin [amittal@hills sbin]$ ls -l init lrwxrwxrwx. 1 root root 22 Jul 28 2021 init -> ../lib/systemd/systemd [amittal@hills sbin]$
Now "systemd" will run the default target by searching the 4 folders mentioned previously.
[amittal@hills sbin]$ cd /etc/systemd/system [amittal@hills system]$ ls -l default.target lrwxrwxrwx. 1 root root 37 May 4 2021 default.target -> /lib/systemd/system/multi-user.target [amittal@hills system]$ [amittal@hills system]$ cat /lib/systemd/system/multi-user.target # SPDX-License-Identifier: LGPL-2.1+ # # This file is part of systemd. # # systemd is free software; you can redistribute it and/or modify it # under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation; either version 2.1 of the License, or # (at your option) any later version. [Unit] Description=Multi-User System Documentation=man:systemd.special(7) Requires=basic.target Conflicts=rescue.service rescue.target After=basic.target rescue.service rescue.target AllowIsolate=yes [amittal@hills system]$
On the hills server the default.target is linked to the multi-user.target. This in turn has dependencies that will be evaluated and the process will repeat. We can also change the default target to boot to by passing in parameters to the kernel at boot time. Recall that the boot loader will load up the kernel . This can be done by editing the boot entry in the boot loader's selection menu. We can also set the parameters permanently by modifying the boot loader's configuration file. Suppose we have to add a "quiet" parameter to the kernel.The below steps apply to Grub loader. Press "e" when the menu shows up and add the parameter to the line at:
linux /boot/vmlinuz-linux root=UUID=0a3407de-014b-458b-b5c1-848e92a327a3 rw quiet
We can also edit the file:
/boot/grub/grub.cfg
Another way is to edit the file "/etc/default/grub" and change the line:
GRUB_CMDLINE_LINUX_DEFAULT="quiet"
And then regenerate the "grub.cfg" file with the command:
grub-mkconfig -o /boot/grub/grub.cfg
We use the systemctl command to manually manage the services.
ajay@ajkumar08-PC:/etc/default$ systemctl status cron
? cron.service - Regular background program processing daemon
Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: >
Active: active (running) since Sun 2023-03-19 23:26:54 PDT; 15h ago
Docs: man:cron(8)
Main PID: 433 (cron)
Tasks: 1 (limit: 4425)
Memory: 424.0K
CPU: 371ms
CGroup: /system.slice/cron.service
+-433 /usr/sbin/cron -f
We can also do "systemctl start", "systemctl stop" and "systemctl restart" to start, stop and restart the service.
This enables the unit. By default the units are disabled. ajay@ajkumar08-PC:/etc/default$ systemctl enable cron.serviceThe "systemctl" command by itself will list all the active unit files. The command "systemctl list-units --all" will list all the units that have been loaded or an attempt has been made to load them.
View the dependency tree for a unit.
We list the dependencies for the default.target which in turn depends on units like the multi-user.target. ajay@ajkumar08-PC:/etc/default$ systemctl list-dependencies default.target | more default.target ? +-accounts-daemon.service ? +-e2scrub_reap.service ? +-gdm.service ? +-switcheroo-control.service ? +-systemd-update-utmp-runlevel.service ? +-udisks2.service ? +-multi-user.target ? +-anacron.service ? +-avahi-daemon.service ...
Shutdown
We should always shutdown a system gracefully using either a GUI option or from the command line. This ensures that the file system is in a consistent states and data from the RAM is flushed out to the hard drive. Just powering off the system can and will create delays when bringing the system up.The shutdown command is:shutdown option time message
We can use just the command "shutdown" or specify a time at which a message will be sent to all the users. The "shutdown" without any options will power off the system. The time can be one of the following:
- a standard linux time, HH:MM
- +N, where N is the number of minutes from now
- now
$ shutdown -r +5 "Emergency maintenance required. Please log-off"
This states that after 5 minutes the system is going to restart. The "-H' means halt and is legacy with little use. It stops the CPU but the power remains on. We can diagnose it at a hardware level by checking voltage levels as an example. Once we are done we can power off manually. The options are:
--help
Print a short help text and exit.
-H, --halt
Halt the machine.
-P, --poweroff
Power-off the machine (the default).
-r, --reboot
Reboot the machine.
-h
Equivalent to --poweroff, unless --halt is specified.
-k
Do not halt, power-off, reboot, just write wall message.
--no-wall
Do not send wall message before halt, power-off, reboot.
-c
Cancel a pending shutdown. This may be used cancel the effect of an invocation of
shutdown with a time argument that is not "+0" or "now".
As expected, we cannot run the "shutdown" command on the hills server and need to run it on a personal or virtual machine in order to try it out.