1 淺析 Linux 初始化 init 系統

第1部分sysvinit: https://www.ibm.com/developerworks/cn/linux/1407_liuming_init1/
第2部分UpStart: https://www.ibm.com/developerworks/cn/linux/1407_liuming_init2/
第3部分Systemd: https://www.ibm.com/developerworks/cn/linux/1407_liuming_init3/

1.1 SysVInit

1.2 UpStart

1.3 Systemd

1.3.1 基本

1 System單元的概念
系統初始化需要做的事情非常多。需要啟動後臺服務，比如啟動 SSHD 服務；需要做配置工作，比如掛載檔案系統。這個過程中的每一步都被 systemd 抽象為一個配置單元，即 unit。
單元的常見型別:
service unit : 副檔名為.service，用於定義系統類服務
target unit : 檔案擴充套件為.target,用於實現模擬”執行級別”
device unit : 檔案擴充套件為.device,用於定義核心識別的裝置。
mount unit : 檔案擴充套件為.mount,定義檔案系統掛載點，利用logind服務，為使用者的會話程序分配CGroup資源
socket unit : 檔案擴充套件為.socket,用於標識程序間通訊的socket檔案
snapshot unit : 檔案擴充套件為.snapshot,管理系統快照
swap unit : 檔案擴充套件為.swap,用於標識swap裝置
automount unit : 檔案擴充套件為.automount,檔案系統自動掛載裝置
path unit : 檔案擴充套件為.path，用於定義檔案系統中的一個檔案或目錄
timer unit : 檔案擴充套件為.timer,定時器配置單元，用來定時觸發使用者定義的操作，這類配置單元取代了atd、crond等傳統的定時服務

1.3.2 初始化過程

1. 當你開啟電源後電腦所做的第一件事情就是BIOS初始化。BIOS會讀取引導裝置設定，定位並傳遞系統控制權給MBR（假設硬碟是第一引導裝置）。
2. MBR從Grub或LILO載入程式讀取相關資訊並初始化核心。接下來將由Grub或LILO繼續引導系統。如果你在grub配置檔案裡指定了systemd作為引導管理程式，之後的引導過程將由systemd完成。Systemd使用“target”來處理引導和服務管理過程。這些systemd裡的“target”檔案被用於分組不同的引導單元以及啟動同步程序。

[email protected]:~# grep -R systemd /etc/grub.d/* 
/etc/grub.d/10_linux:SUPPORTED_INITS="sysvinit:/lib/sysvinit/init systemd:/lib/systemd/systemd upstart:/sbin/upstart"
/etc/grub.d/20_linux_xen:SUPPORTED_INITS="sysvinit:/lib/sysvinit/init systemd:/lib/systemd/systemd upstart:/sbin/upstart"
 

3. systemd執行的第一個目標是graphical.target。

[email protected]:~# systemctl get-default
graphical.target

檢視執行級別：

[email protected]:~# runlevel
N 5

檔案graphical.target的實際位置是/lib/systemd/system/graphical.target。

[email protected]:~# find / -name "graphical.target"
/lib/systemd/system/graphical.target

graphical.target檔案的內容:

 
[email protected]:~# cat /lib/systemd/system/graphical.target
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=Graphical Interface
Documentation=man:systemd.special(7)
Requires=multi-user.target
Wants=display-manager.service
Conflicts=rescue.service rescue.target
After=multi-user.target rescue.service rescue.target display-manager.service
AllowIsolate=yes

4. 在這個階段，會啟動multi-user.target，而這個target將自己的子單元放在目錄“/etc/systemd/system/multi-user.target.wants”裡。這個target為多使用者支援設定系統環境。非root使用者會在這個階段的引導過程中啟用。防火牆相關的服務也會在這個階段啟動。

[email protected]:~# cat /lib/systemd/system/multi-user.target
[Unit]
Description=Multi-User System
Documentation=man:systemd.special(7)
Requires=basic.target
Conflicts=rescue.service rescue.target
After=basic.target rescue.service rescue.target
AllowIsolate=yes

"multi-user.target"會將控制權交給另一層“basic.target”。 目錄“/etc/systemd/system/multi-user.target.wants”：

[email protected]:~# ls /etc/systemd/system/multi-user.target.wants/
atd.service        cron.service            lxcfs.service           neutron-linuxbridge-agent.service    openstack.service      rsyslog.service           snapd.service
cgmanager.service  libvirt-bin.service     lxd-containers.service  neutron-linuxbridge-cleanup.service  open-vm-tools.service  snapd.autoimport.service  ssh.service
cgproxy.service    libvirt-guests.service  networking.service      nova-compute.service                 remote-fs.target       snapd.refresh.timer       ufw.service

5. "basic.target"單元用於啟動普通服務特別是圖形管理服務。它通過/etc/systemd/system/basic.target.wants目錄來決定哪些服務會被啟動，basic.target之後將控制權交給sysinit.target.

[email protected]:~# cat /lib/systemd/system/basic.target 
[Unit]
Description=Basic System
Documentation=man:systemd.special(7)
Requires=sysinit.target
Wants=sockets.target timers.target paths.target slices.target
After=sysinit.target sockets.target paths.target slices.target tmp.mount

Wants=tmp.mount

6. "sysinit.target"會啟動重要的系統服務例如系統掛載，記憶體交換空間和裝置，核心補充選項等等。

[email protected]:~# cat /lib/systemd/system/sysinit.target
[Unit]
Description=System Initialization
Documentation=man:systemd.special(7)
Conflicts=emergency.service emergency.target
Wants=local-fs.target swap.target
After=local-fs.target swap.target emergency.service emergency.target

sysinit.target在啟動過程中會傳遞給local-fs.target。 7. local-fs.target，這個target單元不會啟動使用者相關的服務，它只處理底層核心服務。這個target會根據/etc/fstab和/etc/inittab來執行相關操作。

[email protected]:~# cat /lib/systemd/system/local-fs.target
[Unit]
Description=Local File Systems
Documentation=man:systemd.special(7)
DefaultDependencies=no
Conflicts=shutdown.target
After=local-fs-pre.target
OnFailure=emergency.target
OnFailureJobMode=replace-irreversibly

參考資料：

1 http://www.ruanyifeng.com/blog/2016/03/systemd-tutorial-commands.html

2 http://www.ruanyifeng.com/blog/2016/03/systemd-tutorial-part-two.html

3 最簡明扼要的 Systemd 教程，只需十分鐘： https://linux.cn/article-6888-1.html

4 走進Linux之systemd啟動過程：https://linux.cn/article-5457-1.html

1.4 systemd & sysvinit

主要差別如下：

（1）預設的 RunLevel（在/etc/inittab檔案設定）現在被預設的 Target 取代，位置是/etc/systemd/system/default.target，通常符號連結到graphical.target（圖形介面）或者multi-user.target（多使用者命令列）。
（2）啟動指令碼的位置，以前是/etc/init.d目錄，符號連結到不同的 RunLevel 目錄 （比如/etc/rc3.d、/etc/rc5.d等），現在則存放在/lib/systemd/system和/etc/systemd/system目錄。
（3）配置檔案的位置，以前init程序的配置檔案是/etc/inittab，各種服務的配置檔案存放在/etc/sysconfig目錄。現在的配置檔案主要存放在/lib/systemd目錄，在/etc/systemd目錄裡面的修改可以覆蓋原始設定。

2 命令啟動Openstck服務

2.1 servcie命令

service  runs  a  System V init script orupstart job in as predictable an environment as possible, removing most environment variables and with the current working directory set to /.

使用如下形式手動重啟、關閉或啟動:

service SCRIPT COMMAND [OPTIONS]
The SCRIPT parameter specifies a System V init script, located in /etc/init.d/SCRIPT, or the name of an upstart job in /etc/init. 
The existence of an upstart job of the  same  name  as  a  script  in  /etc/init.d will cause the upstart job to take precedence over the init.d script.  
The supported values of COMMAND depend on the invoked script.  service passes COMMAND and OPTIONS to the init script unmodified. 
For upstart jobs, start, stop, status, are passed through to their upstart equivalents. 
Restart will call the upstart 'stop' for the job, followed immediately by the 'start', and will exit with the return code of the start command.  
All scripts should support at least the start and stop commands.  

如：

# service nova-api restart

System V init scripts和upstart jobs目錄結構

/etc/init
    The directory containing upstart jobs.

/etc/init.d
    The directory containing System V init scripts.

/etc/rc?.d/
    The directories containing the links used by init and managed by update-rc.d.

/etc/init.d/skeleton
    Model for use by writers of init.d scripts.

/var/lib/sysv-rc/legacy-bootsequence
    Flag indicating the machine is using legacy mode for boot script ordering.

2.2 systemctl

systemctl:   start NAME...                   Start (activate) one or more units
  stop NAME...                    Stop (deactivate) one or more units
  reload NAME...                  Reload one or more units
  restart NAME...                 Start or restart one or more units

[email protected]:~# ps -ef | grep nova
nova      3489     1  0 May19 ?        00:03:01 /usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf --log-file=/var/log/nova/nova-compute.log
[email protected]:~# systemctl stop nova-compute
[email protected]:~# ps -ef | grep nova
root      9624  1821  0 09:19 pts/0    00:00:00 grep --color=auto nova
[email protected]:~# systemctl start nova-compute
[email protected]:~# ps -ef | grep nova
nova      9644     1 99 09:19 ?        00:00:02 /usr/bin/python /usr/bin/nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf --log-file=/var/log/nova/nova-compute.log

3 開機自啟動

OpenStack中的各個元件服務都會以守護程序形式在系統後臺執行，並且是開機自啟動。

3.1 sysvinit

以nova-api為例，指令碼檔案/etc/init.d/nova-api，

[email protected]:~# ls /etc/init.d/nova-api 
/etc/init.d/nova-api
[email protected]:~# cat /etc/init.d/nova-api 
#!/bin/sh
### BEGIN INIT INFO
# Provides:          nova-api
# Required-Start:    $network $local_fs $remote_fs $syslog
# Required-Stop:     $remote_fs
# Should-Start:      postgresql mysql keystone rabbitmq-server ntp
# Should-Stop:       postgresql mysql keystone rabbitmq-server ntp
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Nova API server
# Description:       Frontend Nova API server
### END INIT INFO

# Author: Julien Danjou <[email protected]>

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="OpenStack Compute API"
PROJECT_NAME=nova
NAME=${PROJECT_NAME}-api
#!/bin/sh
# The content after this line comes from openstack-pkg-tools
# and has been automatically added to a .init.in script, which
# contains only the descriptive part for the daemon. Everything
# else is standardized as a single unique script.

# Author: Thomas Goirand <[email protected]>

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/sbin:/usr/sbin:/bin:/usr/bin

if [ -z "${DAEMON}" ] ; then
	DAEMON=/usr/bin/${NAME}
fi
PIDFILE=/var/run/${PROJECT_NAME}/${NAME}.pid
if [ -z "${SCRIPTNAME}" ] ; then
	SCRIPTNAME=/etc/init.d/${NAME}
fi
if [ -z "${SYSTEM_USER}" ] ; then
	SYSTEM_USER=${PROJECT_NAME}
fi
if [ -z "${SYSTEM_USER}" ] ; then
	SYSTEM_GROUP=${PROJECT_NAME}
fi
if [ "${SYSTEM_USER}" != "root" ] ; then
	STARTDAEMON_CHUID="--chuid ${SYSTEM_USER}:${SYSTEM_GROUP}"
fi
if [ -z "${CONFIG_FILE}" ] ; then
	CONFIG_FILE=/etc/${PROJECT_NAME}/${PROJECT_NAME}.conf
fi
LOGFILE=/var/log/${PROJECT_NAME}/${NAME}.log
if [ -z "${NO_OPENSTACK_CONFIG_FILE_DAEMON_ARG}" ] ; then
	DAEMON_ARGS="${DAEMON_ARGS} --config-file=${CONFIG_FILE}"
fi

# Exit if the package is not installed
[ -x $DAEMON ] || exit 0

# If ran as root, create /var/lock/X, /var/run/X, /var/lib/X and /var/log/X as needed
if [ `whoami` = "root" ] ; then
	for i in lock run log lib ; do
		mkdir -p /var/$i/${PROJECT_NAME}
		chown ${SYSTEM_USER} /var/$i/${PROJECT_NAME}
	done
fi

# This defines init_is_upstart which we use later on (+ more...)
. /lib/lsb/init-functions

# Manage log options: logfile and/or syslog, depending on user's choosing
[ -r /etc/default/openstack ] && . /etc/default/openstack
[ -r /etc/default/$NAME ] && . /etc/default/$NAME
[ "x$USE_SYSLOG" = "xyes" ] && DAEMON_ARGS="$DAEMON_ARGS --use-syslog"
[ "x$USE_LOGFILE" != "xno" ] && DAEMON_ARGS="$DAEMON_ARGS --log-file=$LOGFILE"

do_start() {
	start-stop-daemon --start --quiet --background ${STARTDAEMON_CHUID} --make-pidfile --pidfile ${PIDFILE} --chdir /var/lib/${PROJECT_NAME} --startas $DAEMON \
			--test > /dev/null || return 1
	start-stop-daemon --start --quiet --background ${STARTDAEMON_CHUID} --make-pidfile --pidfile ${PIDFILE} --chdir /var/lib/${PROJECT_NAME} --startas $DAEMON \
			-- $DAEMON_ARGS || return 2
}

do_stop() {
	start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE
	RETVAL=$?
	rm -f $PIDFILE
	return "$RETVAL"
}

do_systemd_start() {
	exec $DAEMON $DAEMON_ARGS
}

case "$1" in
start)
	init_is_upstart > /dev/null 2>&1 && exit 1
	log_daemon_msg "Starting $DESC" "$NAME"
	do_start
	case $? in
		0|1) log_end_msg 0 ;;
		2) log_end_msg 1 ;;
	esac
;;
stop)
	init_is_upstart > /dev/null 2>&1 && exit 0
	log_daemon_msg "Stopping $DESC" "$NAME"
	do_stop
	case $? in
		0|1) log_end_msg 0 ;;
		2) log_end_msg 1 ;;
	esac
;;
status)
	status_of_proc "$DAEMON" "$NAME" && exit 0 || exit $?
;;
systemd-start)
	do_systemd_start
;;  
restart|force-reload)
	init_is_upstart > /dev/null 2>&1 && exit 1
	log_daemon_msg "Restarting $DESC" "$NAME"
	do_stop
	case $? in
	0|1)
		do_start
		case $? in
			0) log_end_msg 0 ;;
			1) log_end_msg 1 ;; # Old process is still running
			*) log_end_msg 1 ;; # Failed to start
		esac
	;;
	*) log_end_msg 1 ;; # Failed to stop
	esac
;;
*)
	echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload|systemd-start}" >&2
	exit 3
;;
esac

exit 0

3.2 upstart

Upstartinit是基於事件驅動的服務啟動機制，可以使多個系統任務在保持依賴關係的前提下併發啟動。 通過/etc/init下的一系列 *.conf 配置檔案來指定各種系統服務的依賴關係（啟動時機）。系統啟動時，upstart主程序/sbin/init會解析這些配置檔案，按照指定的依賴關係併發啟動各種服務與應用。
將自己的程式加入到系統的服務啟動機制中，可以讓其開機自啟動。只需要在/etc/init下增加xxx.conf配置檔案即可，xxx是service的服務名字。
兩個概念：
1. init: upstart主程序，是Linux系統中的“應用程式管理器”，是其他所有程序的源頭（PID為1），它會讀取配置檔案，處理各種服務和應用程式的依賴關係，根據事件（訊號）來啟動這些功能與服務，並動態地進行管理。
2.initctl:upstart事件管理器，可以被應用程式程序用來通知init哪些事件（訊號）發生。
nova-api的配置/etc/init/nova-api.conf：

[email protected]:~# ls /etc/init/nova-api.conf 
/etc/init/nova-api.conf
[email protected]:~# 
[email protected]:~# cat /etc/init/nova-api.conf 
description "OpenStack Compute API"
author "Thomas Goirand <[email protected]>"

start on runlevel [2345]
stop on runlevel [!2345]

chdir /var/run

respawn
respawn limit 20 5
limit nofile 65535 65535

pre-start script
	for i in lock run log lib ; do
		mkdir -p /var/$i/nova
		chown nova /var/$i/nova
	done
end script

script
	[ -x "/usr/bin/nova-api" ] || exit 0
	DAEMON_ARGS=""
	[ -r /etc/default/openstack ] && . /etc/default/openstack
	[ -r /etc/default/$UPSTART_JOB ] && . /etc/default/$UPSTART_JOB
	[ "x$USE_SYSLOG" = "xyes" ] && DAEMON_ARGS="$DAEMON_ARGS --use-syslog"
	[ "x$USE_LOGFILE" != "xno" ] && DAEMON_ARGS="$DAEMON_ARGS --log-file=/var/log/nova/nova-api.log"

	exec start-stop-daemon --start --chdir /var/lib/nova \
		--chuid nova:nova --make-pidfile --pidfile /var/run/nova/nova-api.pid \
		--exec /usr/bin/nova-api -- --config-file=/etc/nova/nova.conf ${DAEMON_ARGS}
end script

upstart init啟動服務在init程序起來後會讀取/etc/init/中的服務程式配置檔案。/etc/init.d下的執行指令碼名字與/etc/init下的配置名字對應。/etc/init/neutron-server.conf與/etc/init.d/neutron-server對應。如果/etc/init.d/neutron-server被刪除，也不會影響service neutron-server start的啟動。服務啟動的某些配置完全可以在/etc/init/xxx.conf中增加。 實驗：Ubuntu14.04中，當把/etc/init/nova-compute.conf檔案移除後，重啟後，nova-compute服務不能啟動 實驗：Ubuntu14.04中，當把/etc/init.d/nova-compute檔案移除後，重啟後，nova-compute服務正常啟動

3.3 systemd

1 ubuntu16.04系統中openstack服務自啟動配置存放在/etc/systemd/system/multi-user.target.wants/，檢視

[email protected]:~# ls /etc/systemd/system/multi-user.target.wants/
neutron-linuxbridge-agent.service    openstack.service      rsyslog.service           snapd.service
neutron-linuxbridge-cleanup.service  open-vm-tools.service  snapd.autoimport.service  ssh.service
nova-compute.service                 remote-fs.target       snapd.refresh.timer       ufw.service

2 檢視nova-compute.service內容：

[email protected]:~# cat /etc/systemd/system/multi-user.target.wants/nova-compute.service 
[Unit]
Description=OpenStack Compute
After=libvirt-bin.service postgresql.service mysql.service keystone.service rabbitmq-server.service ntp.service neutron-ovs-cleanup.service 

[Service]
User=nova
Group=nova
Type=simple
WorkingDirectory=/var/lib/nova
PermissionsStartOnly=true
ExecStartPre=/bin/mkdir -p /var/lock/nova /var/log/nova /var/lib/nova
ExecStartPre=/bin/chown nova:nova /var/lock/nova /var/lib/nova
ExecStartPre=/bin/chown nova:adm /var/log/nova
ExecStart=/etc/init.d/nova-compute systemd-start
Restart=on-failure
LimitNOFILE=65535
TimeoutStopSec=15

[Install]
WantedBy=multi-user.target

nova-compute.service被呼叫後，會執行etc/init.d/nova-compute指令碼 3 檢視nova-compute的自啟動指令碼和配置都存在：

[email protected]:~# ls /etc/init/nova-compute.conf 
/etc/init/nova-compute.conf
[email protected]:~# ls /etc/init.d/nova-compute 
/etc/init.d/nova-compute

因為沒有使用upstart，所以/etc/init/nova-compute.conf配置是不起作用的。
實驗：Ubuntu16.04中，當把/etc/init/nova-compute.conf檔案移除後，重啟後，nova-compute服務正常啟動 實驗：Ubuntu16.04中，當把/etc/init.d/nova-compute檔案移除後，重啟後，nova-compute服務不能啟動

4 取消開機自啟動

4.1 sysvinit

4.2 upstart

nova-api.conf為例，只需要把start on註釋掉就好了。不要刪除conf檔案，這樣以後仍然可以用start/stop這些命令來控制它的狀態。

4.3 systemd

方法1：移除檔案etc/init.d/nova-compute 方法2：修改檔案/etc/systemd/system/multi-user.target.wants/nova-compute.service

5 nova-api呼叫執行

(1) /etc/init/nova-api.conf關鍵程式碼如下：

exec start-stop-daemon --start --chdir /var/lib/nova \
		--chuid nova:nova --make-pidfile --pidfile /var/run/nova/nova-api.pid \
		--exec /usr/bin/nova-api -- --config-file=/etc/nova/nova.conf ${DAEMON_ARGS}

其實就是用start-stop-daemon去呼叫/usr/bin/nova-api

(2) 檢視/usr/bin/nova-api

[email protected]:~# cat /usr/bin/nova-api 
#!/usr/bin/python
# PBR Generated from u'console_scripts'

import sys

from nova.cmd.api import main

if __name__ == "__main__":
    sys.exit(main())

其中呼叫了nova/cmd目錄下的api.py檔案中的main函式
(3) 檢視api.py檔案

[email protected]:~# python
Python 2.7.6 (default, Oct 26 2016, 20:30:19) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import nova
>>> print nova.__path__
['/usr/lib/python2.7/dist-packages/nova']

到目錄/usr/lib/python2.7/dist-packages/nova/cmd下：

[email protected]:/usr/lib/python2.7/dist-packages/nova/cmd# ll api*py
-rw-r--r-- 1 root root 1514 Nov 22 11:11 api_ec2.py
-rw-r--r-- 1 root root 1781 Nov 22 11:11 api_metadata.py
-rw-r--r-- 1 root root 1481 Nov 22 11:11 api_os_compute.py
-rw-r--r-- 1 root root 1846 Nov 22 11:11 api.py