1.0 Upstart Semantics
Initial Draft
What is this document
This document is intended to completely specify the behavior of Upstart. It provides an internal mechanism by which Upstart might operate, but is only specific enough in doing so to explain all desired externally observed behavior. You can think of this document as a good picture of Upstart to have in your head while operating it. It may not match the code exactly, but if you don't look at the code directly, you shouldn't be able to tell.
This document comes from Scott's talk at 2009 FOSDEM, and from his discussion of implementation on IRC. It bears some structural resemblance to Upstart 0.5, and could be looked at as a rough recommendation for the internal design of Upstart 1.0, though this is not its primary purpose.
This document is meant to be a formal specification for behavior. To be useful, the system described here should have a known behavior under any' circumstances. It is not appropriate for this document to specify undefined behavior, or to imply such by omission. If this is achieved, this document can serve as a foundation for black-box testing. Upstart is correctly implemented if and only if its behavior matches this document.
Key concepts
Upstart is built on a few primitive concepts that will be described in this section.
Several of these concepts are discussed in an object oriented fashion, where an entity is described, its properties are listed, and a series of operations (methods) which can be performed on it are given. Property lists are formatted as follows:
property - Description of the property
another - Another description
- Description of other information available in the object which we do not break down into properties
Methods are listed as follows:
do_something - Description of what happens
Takes: A particular argument needed to perform this operation
Takes: Another argument
Returns: Information output after performing the operation
When properties and methods are referred to in the text, they will appear in monospace, i.e. do_something. If disambiguation is necessary, the entity claiming the property or method will appear after in parentheses, i.e. do_something (Example). When the names of particular entities themselves is mentioned in the text, it will be capitalized and in monospace, i.e. Example.
Environment variables
Several objects in upstart carry environment variables. They are used to specify certain properties of services and to pass along information to processes. When this document refers to a set of environment variables, it is expected to be a series of key/value pairs where each key maps to only one value.
We may also mention environment patterns. These are sets of key/expression pairs where the expression specifies a range or type of value rather than a value itself. A set of environment variables matches an environment pattern if for each key in the pattern, it possesses a value that matches the expression.
Events
An event in Upstart is simply a particular point in time. It can be "3pm Thursday" or "The moment we first found out a network card was plugged in." In the case of the first example, Upstart can detect that "3pm Thursday" has arrived by watching the system time. In the case of the second, some external application (in this case likely device kit) would have to notify Upstart that the event occurred.
A particular occurrence of an event may have a set of environment variables associated with it indicating details of that occurrence. In our network card example, it might contain the name of the new network device.
Classes
A Class is a basic definition of a service which Upstart can launch and operate on. Every class possesses:
name - A name such as "apache" or "mingetty"
dependencies - A list of pairs of one Class and one environment pattern. Each pairing is a "dependency." An Instance is said to satisfy a dependency if it is of the Class in the dependency and its environment matches the environment pattern.
start_on - A list of events which cause this service to start
environment - A default set of environment variables
- Information on how to start an instance of the service, such as the path to the binary and rlimit settings to run it under (This may not be applicable. See below)
A Class is a sort of template configuration for a service. For a given class, one or more Instances (see below) of the defined service may be running.
In addition to services, some classes may represent a state or condition which may be true or false on the system. These classes behave identically, but are associated with no process or binary.
Operations on classes
The following operations can be performed by the user or by Upstart internally on a class:
instantiate - Create an instance of this class (see below)
Takes: A set of environment variables. These are merged with environment (Class) to form the contents of environment (Instance)
Takes: A list of instances in the running state, one per item in dependencies, which satisfy each dependency. The list cannot be incomplete.
Returns: A handle for the new instance
start - Find one existing instance of this class which is not in the running state and start it.
Returns: A handle for the started instance
stop - Stop all running instances of a class
Instances
An instance represents one particular case of a given service which is ready to be started or run. For example, you may have a Class for a getty service, but you may have multiple Instances of that service running on different TTYs.
Every instance has the following properties
class - the Class to which this instance belongs
depends_on - a list of other Instance objects which satisfy dependencies of class
environment - Environment variables for this instance. These are passed to the process associated with the instance, if any
state - One of:
- Starting
- Running
- Stopping
- Waiting
start_on - A list of events which will cause this instance to move toward the Running state when they occur
- Information necessary to manage system processes running due to this instance (PID etc.)
States
The state property indicates what state the Instance is in. The state of the Instance explains what is going on with it at that moment. If the Instance is in the Waiting state, then no service is running and the instance cannot satisfy any dependencies. If the instance is in the Running state, then the opposite is true. Starting and Stopping behave like Waiting except they indicate that state is about to become Running or Waiting respectively. It is possible for dependency information to be briefly inconsistent while the Instance is in Starting or Stopping. i.e. An Instance may depend on another Instance which is in the Stopping state. Upstart is required to resolve these inconsistencies before progressing to the Running or Waiting states. No Instance should remain in the Starting or Stopping state indefinitely.
state is only allowed to be assigned certain values, depending on what its current value is. Valid changes to state include:
Waiting -> Starting - When something invokes start (Instance) or an event mentioned in start_on occurs, or when start_on is an empty list and no other instance of class is running
Starting -> Running - When all dependencies are consistent and no failures were encountered while starting any services correlated to this Instance
Starting -> Waiting - When all dependencies are consistent and failures were encountered while starting services correlated to this Instance
Running -> Stopping - When something invokes stop (Instance) or a service correlated to this Instance has failed
Starting -> Stopping - When something invokes stop (Instance) or a service correlated to this Instance has failed
Stopping -> Starting - When something invokes start (Instance) or an event mentioned in start_on occurs
Stopping -> Waiting - When all dependencies are consistent
Generally speaking an instance will make one of these transitions whenever it is legal.
Operations on instances
The following actions are available for instances
start - Cause the instance to move from the Waiting or Stopping states into the Starting state
stop - Cause the instance to move from the Starting or Running states into the Stopping state
restart - Equivalent to calling stop, waiting for state to become Waiting, then calling start
Operation
The Queue
When an event occurs or something attempts to call a method on one of Upstart's objects, this action is placed at the tail of a Queue. Upstart processes actions from the head of the queue, one at a time. This has an effect of serialization; we do not have to consider the idea of two events or method calls happening at once. During operation, Upstart may occasionally add one or more events to the head of the queue to be processed immediately.
Leaving the Starting and Stopping states
When an Instance enters the Starting or Stopping state, it may then trigger one or more events on the host system (launching or killing a process for example), the completion of which is required for transition into the Waiting or Running state. Upstart handles these events asynchronously. Items are added to the queue when an Instance should be allowed to safely enter the Running or Waiting states.
Garbage Collection
Garbage collection occurs any time an Instance enters the Waiting state, before the next item in the queue is processed. Garbage collection destroys certain instances based on the following criteria:
If there is more than one instance in the Waiting state with identical values for class and depends_on, all but one will be destroyed
If an instance refers to any other instance in its depends_on list which is in the Waiting state it is destroyed
Re-population
Re-population occurs immediately after Upstart startup, before the first item in the queue is processed, or immediately after an Instance enters the Running state. Re-population creates new instances based on the following criteria:
- No new instance may be created that would immediately be swept up by garbage collection
Any valid set of services in the Running state (see below) which satisfy all of dependencies (Class) will be passed to instantiate for that class to create a new Instance
Valid Sets of Instances for Dependency Satisfaction
A set of instances can be used to satisfy the dependencies of a new Instance if:
There is exactly one Instance satisfying each member of dependencies for the target class
All of the Instance objects in question are in the Running state
If the class of any of the Instance objects specifies a dependency which may be satisfied by any of the other Instance objects in the set in question, the former Instance mentions the later in its depends_on list
Handling of Stop conditions
If Upstart processes an item from the queue, and determines that that item would cause an Instance to move out of the Running state, and any other Instance depends_on that first Instance, the item is immediately returned to the head of the queue. Upstart then prepends stop calls for each dependent instance to the head of the queue.