Debugging Upstart
The document describes some helpful techniques for debugging. These methods can be used for new Upstart development, porting Upstart to a new Linux distribution, or authoring job definition files.
Increase Upstart Verbosity
Add a "--debug" command line to the kernel options after the init=/opt/upstart/sbin/init command. The "--debug" option causes Upstart to display additional messages to the console.
Add Persistent Log Entries to Job Definition Files
It may be helpful to keep a persistent record of what happens during the boot process. By writing to a file in /tmp we can watch boot messages after a successful boot. If the boot isn't successful, we can review the /tmp file after rebooting into a known good boot.
echo "Debug Message" >> /tmp/upstart
Make sure to delete the /tmp files after each debug boot to not confuse two boots.
This is not clear enough: how do I activate the persistant record? What do I have to do? Does upstart check at the startup if there is a file /tmp/upstart with the content "Debug Message"?
Initctl Commands
Once your system is up, Upstart provides useful commands for debugging jobs. "Initctl events" displays events as they are emitted in real time. Events can be generated from services or user generated from the "initctl emit <event>" command.
"Initctl jobs" displays verbose information in real time from Upstart about the actions taken on jobs.
Both commands do not display history, they only display activities as they occur. So when you first start the command you will not see anything happening. Both commands are exited with a CTRL+C.
"Initctl list" shows a complete listing of jobs and their current status. "Initctl status <job>" shows the same status as "initctl list", but for only a single job.
Boot Problems
If there is ever a problem booting, you can always change your init on the kernel command line. Depending on your distribution, access your kernel command line and add:
init=/bin/sh
Even if you have replaced init with Upstart's init, you can boot with a minimal setup. From the shell prompt you can modify configuration files, or replace binaries as needed.
-- CategoryDoc