upstart is replaced in RedHat7 with systemd. upstart is quite confusing, and since it's tenure is limited, read this section lightly. For those of you who use Ubuntu, however, upstart is currently heavily used there. Even this will be changing - Ubuntu, upstart's maintainer, will switch to systemd with Ubuntu 14.10, due to come out in October 2014.

You will not be tested on upstart, nor will it be covered in class.

upstart is the replacement for System5 init that is installed on Redhat 6. We have already discussed System5 init and how it uses the rc scripts and subsystems to initialize the system. There are several shortcomings with this scheme that upstart addresses:

• the System5 init procedure executes subsystems sequentially. Although many subsystems could be run in parallel, sequential execution is required to enforce subsystem dependencies which are imposed by the simplistic link numbering system. This means startup is slower than it needs to be.
  

• the only way to get System5 init to do something is via the inittab, and the only way to trigger this is to change the runlevel. If a daemon (started by an upstart job) is to orchestrate system initialization and state, it needs to respond to system changes, not just to externally-imposed runlevels. (As an example, if daemon A is responsible for hardware changes and daemon B wants to be informed of them, upstart events can be used, allowing daemon A to communicate the state change to daemon B via upstart.)
  

• Linux systems have been rapidly moving to an event-based system using such facilities as dbus, inotify, and gconf. (Unfortunately, these topics have been removed from this class due to time-constraits.) init needed to work in this same way and be more dynamic.

upstart deals with these issues by emitting and responding to events. These events are configured to control jobs it manages. Each event has a name. Some also have parameters.

We will categorize events as three types for our discussion

• the startup event, emitted by upstart when it is finished initializing and it is ready to start its jobs.

• standard job events emitted by upstart. The most common use of upstart is to start or stop a job. start and stop are called 'job goals'. Standard job events indicate what stage of progress to the job goal the job is in in its life-cycle. There are four job events: starting, started, stopping, and stopped. upstart emits these events for every job that it manages. Their emission is internal to upstart. In other words, upstart emits the events on its own, without being told to do so by another job.
  

• events defined by a job (job-defined events) and emitted by upstart at the direction of that job (or by the administrator). One such event is the runlevel event. When telinit is told to change the runlevel, it tells upstart to emit the runlevel event, which is used to simulate runlevels (see below). The runlevel event takes an argument - the target runlevel. One job that is started by this event "exports" a variable $RUNLEVEL set to the current runlevel. ('export' here means 'make this variable available to all events of this job'.) Although a group of jobs may know about a single job-defined event, they are not standard. They are only issued by specific code that tells upstart to emit them, as contrasted with standard job events.
  

The emission of any event may trigger actions in other jobs if they are so configured. If multiple jobs are configured to start on an event, upstart is free to start the jobs in any order, possibly in parallel.

A detailed discussion of upstart and how it works is beyond this course. We will, however, introduce the startup event and a few job-specific events and see how upstart manages jobs in response to these events. We will also see how jobs can enforce dependencies using these events.

The startup event

As indicated previously, the startup event is issued by upstart when it is ready to start jobs. It then starts all jobs (see below) that are configured to start on the startup event. These jobs tend to be the master jobs that are responsible for getting the initialization ball rolling. The jobs started due to the startup event [tell upstart to] emit events that are used to start other jobs.

upstart jobs

upstart jobs are kept in the /etc/init directory. The filename indicates the name of the job, with the extension .conf They are not shell scripts, so they do not have to be executable.

Each job contains

• which event(s) start and stop the job

• the code to be executed when upstart 'starts' the job (when upstart's job goal is changed to start for the job). This may be an exec command (for a command to run) or a script section containing shell commands. This code is executed between emitting the starting and started events for the job. There are a few directives in this code that control how it is run and what to do if it exits (see under "Dry run" below).
  Normally when this main job code (the "start code") exits, upstart changes its job goal to stop and proceeds with the sequence of stop events ('stops' the job). In other words, upstart wants to complete the job cycle. Left on its own, it will pause after the job is started until the code that it started is finished (in the case of a daemon, until the daemon exits.) Then it executes the stop code to clean up any dependencies. Of course, upstart could be told independently to stop the job at any time.
  

• work to be done at the stages of starting and stopping. These include optional pre-start code (executed when the starting event is emitted), post-start code (executed when the started event is emitted). It also includes optional pre- and post- stop code (see below).
  

Here is the sequence that occurs when upstart starts job Y:

• It emits the starting event for job Y
  

• It runs Y's pre-start code (if any)

• It starts a new process and runs Y's main code. 

• normally, when Y's main code is started, upstart is allowed to continue with what it was doing. (If Y was started due to an event emitted by another upstart job X, upstart continues running job X in parallel with Y.) This is what happens unless the main code of Y includes a task directive.
  

• if Y's main code includes a task directive, the event(s) that caused Y to start are blocked until Y transitions back to stopped. Let's assume that Y was started due to X's starting event. This would be indicated in Y's upstart job as

start on starting X

• if the main code includes a respawn directive, the main code will be re-executed if it exits unless the job's goal has been changed to stop.
  

• Y's post-start code is executed (if any).
  

• upstart emits the started event for job Y.

Here is the sequence that occurs when upstart stops job Y:

• If there is a process associated with job Y, upstart

• runs Y's pre-stop code (if any)

• kills the process using a sequence of TERM and KILL signals

• It emits the stopping event for job Y
  

• It runs the Y's post-stop code (if any)

• It emits the stopped event for job Y.

The runlevel event

You should only change the runlevel by invoking telinit. Although initctl could be used to emit the runlevel event, doing this will not record the runlevel correctly in /var/log/wtmp. This will cause confusion in the rc scripts and inconsistency in the output of runlevel and who -r

A "simple" example

This is a lot to soak in and I know it doesn't make much sense. So let's look at a contrived simple example. Here is our job (the contents of the file /etc/init/xxx.conf, which I created)

As you can see, we have defined our own event, foobar. (This makes our script only run when we say so.) When we emit the event using upstart:

# initctl emit foobar

initctl returns immediately. If we examine the file /tmp/xxx.out we find

# cat /tmp/xxx.out
xxx pre-start
xxx main code started
xxx post-start

10 seconds or so later:

Let's add a second job, /etc/init/yyy.conf:

# cat yyy.conf
start on starting xxx
script
sleep 1
echo "yyy ran" >> /tmp/xxx.out
end script

Our new job, yyy, starts when upstart emits the starting event for xxx. Now when we emit foobar (and wait 11 seconds) (although initctl still returns immediately)

# initctl emit foobar

It is not obvious that xxx's main code and yyy's code ran in parallel. You can see this, however, if we add the task directive to yyy, forcing yyy to run to completion before xxx is allowed to continue.

# cat yyy.conf
start on starting xxx
task
script
sleep 1
echo "yyy ran" >> /tmp/xxx.out
end script

And, again, emit foobar and wait 11 seconds:

You can see that yyy was inserted and ran to completion before xxx's main (or pre-start) code was run.

Notice that there is no pre-stop output from xxx, since xxx was not explicitly stopped. If we emit foobar and immediately issue a stop directive for xxx (before the 11 seconds have elapsed):

We can now see how upstart jobs use of events can impose order without having to rely on some ad-hoc filename order, as are used in subsystems. This allows jobs that can be run in parallel, to be run in parallel. Anyone who has compared the speed of booting Redhat systems pre- and post- version 6 sees some difference. And in Ubuntu, which has moved a considerable number of subsystems to upstart jobs, booting is very fast indeed.

A few real jobs

At the time of this writing, our systems had very little placed under the control of upstart, but there are two sample jobs that perform a function we are acquainted with:

The first is the job rcS.conf.

It is one of the few jobs under control of upstart that responds to the startup event. The line that configures this is

start on startup

This says that rcS.conf will be started (the job goal event 'start' will be processed) by upstart when it initializes the system. When the job is started, upstart emits the starting event. Since there is no pre-start code, it runs the exec or script section, which in this case is

exec /etc/rc.d/rc.sysinit

if I was started by upstart with the startup event

if I was just booted with instructions to go to single-user mode, use S as the runlevel
else
extract the runlevel from /etc/inittab

run telinit with the new runlevel.

telinit then has upstart emit the runlevel event.

The second job uses the runlevel event. It is the job rc.conf. Here is a copy of that job so we can analyze it:

start on runlevel [0123456]

stop on runlevel [!$RUNLEVEL]

task

export RUNLEVEL
console output
exec /etc/rc.d/rc $RUNLEVEL

This job starts on a runlevel event. When started, it simply starts the rc script, giving it the new runlevel.

The stop on directive is curious. It is there to cause upstart to stop rc if it is currently running when the runlevel changes again! ($RUNLEVEL contains the current runlevel. The argument to the runlevel event is the new runlevel.)

We will investigate a few more jobs in the Exercises. You should peruse the /etc/init directory and see if you can figure out how they interact. The important jobs are rc, rcS, start-ttys, tty, init-system-dbus and prefdm.

A little more verbosity

Here is some log information output by upstart showing how the job rc progresses through the various states in upstart. I have annotated it to show where the sections of code in the job are executed and when the actual events are emitted:

Controlling jobs

As we have seen, you can control jobs yourself by using initctl. There are a lot of commands, but here are the introductory ones:

the first batch of initctl commands are similar to those of service (except the arguments are annoyingly reversed):

initctl start job

initctl stop job

initctl status job

initctl restart job

the next batch are a bit different:

initctl emit event

initctl reload job - send the job a HUP signal

initctl list - list all jobs and their status

The X window manager

The prefdm.conf initctl job on Redhat controls the X window manager. If you need to stop or restart it you should use the appropriate initctl command. Remember, this will kill all GUI applications and logout all users logged in under the GUI. It will return to a new login screen. If someone leaves the console logged in, however, this will log them (and anyone else who is logged in under the GUI) out.

Users of previous Redhat and other linux systems may recall being able to use the keystroke control-alt-backspace (CNTL-ALT-BS) to kill the window manager. This is still available as an option, but it is configurable on a user-by-user basis and it is somewhat difficult to find: it is in System->Preferences->Keyboard->Layouts->Layout Options

Since this is configurable on a user-by-user basis, it will only be available if the currently logged-in user so configures it! This makes it virtually unusable as a forced-logout technique for a user who leaves themself logged in. The only real solution is:

• open a virtual console using CNTL-ALT-F2 (or 3 or 4 ...)
• become root
• issue the command initctl restart prefdm

Note that you will be switched back to the GUI console, but your virtual console will still be logged in as root! Make sure you switch back to the virtual console and log yourself out. To avoid this, you may want to add ;exit to the end of your initctl command: 

• initctl restart prefdm; exit