如果我們需要動態除錯和開發BeeGFS，那麼就需要自己手動編譯了，而且能夠用最簡單的方式執行起來，下面就講解自己探索和分析的過程。

• 參考文件：

# https://git.beegfs.io/pub
# https://www.beegfs.io/wiki/BuildFromSources

初始化環境

• 安裝依賴的軟體包：

$ yum install -y libuuid-devel libibverbs-devel librdmacm-devel libattr-devel redhat-rpm-config rpm-build xfsprogs-devel cppunit cppunit-devel zlib-devel openssl-devel sqlite sqlite-devel ant gcc-c++ gcc redhat-lsb-core java-devel
 

獲取官方原始碼

• 獲取指定版本的原始碼：

$ git clone https://git.beegfs.io/pub/v6.git beegfs-v6
$ cd beegfs-v6

$ git tag -l
6.1
6.10
6.11
6.12
6.13
6.14
6.15
6.16
6.17
6.18
6.19
6.2
6.3
6.4
6.5
6.6
6.7
6.8
6.9

$ git checkout -b v6.18 6.18

• 進行簡單的檢視和驗證：

$ git show 6.18 | head -n12
tag 6.18
Tagger: Bernd Lietzow <[email protected]>
Date:   Mon Mar 12 16:09:35 2018 +0100

created tag 6.18

commit aee03250ea19502952d2f187e73134996abaec5b
Author: Bernd Lietzow <[email protected]>
Date:   Mon Mar 12 16:09:13 2018 +0100

    updated to release 6.18


$ git branch -av
  master                4a69532 updated to release 6.19
* v6.18                 aee0325 updated to release 6.18
  remotes/origin/HEAD   -> origin/master
  remotes/origin/master 4a69532 updated to release 6.19
 

分析Makefile

• 檢視他的Makefile，BeeGFS自己實現了一個簡單的Makefile工程模板，所有的子模組都使用這一套模板來變異，十分簡單高效：

$ make -j$(nproc) -C beegfs_meta/build help
[VER]   BEEGFS_VERSION="19.5-git8" BEEGFS_VERSION_CODE=18023688
make: Entering directory `/root/beegfs-v6/fhgfs_meta/build'
Optional arguments:
  BEEGFS_DEBUG=1             Enables debug information and symbols
  BEEGFS_DEBUG_OPT=1         Enables internal debug code, but compiled
                             with optimizations
  CXX=<compiler>             Specifies a c++ compiler
  DISTCC=distcc              Enables the usage of distcc
  V=1                        Print command lines of tool invocations
  BEEGFS_COMMON_PATH=<path>  Path to the beegfs_common directory
  BEEGFS_OPENTK_PATH=<path>  Path to the beegfs_opentk_lib directory
  BEEGFS_THIRDPARTY_PATH=<path>
                             Path to the beegfs_thirdparty directory

Targets:
  all (default)     build only
  clean             delete compiled files
  help              print this help message
  rpm               create an rpm package
  deb               create debian package file
make: Leaving directory `/root/beegfs-v6/fhgfs_meta/build'

$ cat beegfs_meta/build/Makefile
# This is the beegfs_meta makefile.
# It creates an ELF executable.
# 
# Use "make help" to find out about configuration options.

CAN_PACKAGE := yes

include $(or $(root-dir),../..)/build/Makefile

$(call build-executable,\
   beegfs-meta,\
   $(shell find ../source -iname '*.cpp'),\
   opentk common cppunit dl)

# enable special reference DirInode debug code
ifneq ($(BEEGFS_DEBUG_RELEASE_DIR),)     # extra release dir debugging
   CXXFLAGS += -DBEEGFS_DEBUG_RELEASE_DIR
endif

$ cat build/Makefile
...
override V := $(if $V,,@)

all:

...

rpm:
	./make-rpm

...

# build-executable
#
# define a new executable for the build
# arguments:
#  #1: name of the executable
#  #2: sources
#  #3: required libraries
build-executable = $(eval $(call -build-executable-fragment,$(strip $1),$2,$3))
define -build-executable-fragment
all: $1

CLEANUP_FILES += $1 $(addsuffix .o,$2) $(addsuffix .d,$2)

$(addsuffix .o,$2): CXXFLAGS += \
   $(foreach lib,$3,$(call resolve-dep-cflags,$(lib)))

$1: LDFLAGS += \
   -Wl,--start-group $(foreach lib,$3,$(call resolve-dep-ldflags,$(lib))) -Wl,--end-group

$1: $(addsuffix .o,$2) $(foreach lib,$3,$(call resolve-dep-deps,$(lib)))
	@echo "[LD]	$$@"
	$$V$$(CXX) -o $$@ $(addsuffix .o,$2) $$(LDFLAGS)

-include $(addsuffix .d,$2)
endef
 

分析RPM指令碼

• 檢視他構建RPM包的方法，直接打包原始碼目錄，並使用一個RPM SPEC檔案模板，使用sed動態生成一個SPEC文來進行構建：

$ cat beegfs_meta/build/make-rpm 
#!/bin/bash 

NAME=beegfs-meta
SPEC_FILE=${NAME}.spec

source ../../beegfs-version

arch=`uname -m`
if [[ "$arch" == "x86_64" ]]; then
	BUILDARCH="$arch"
elif [[ "$arch" =~ 86 ]]; then
	BUILDARCH="i686"
else
	BUILDARCH=$arch
fi

CONCURRENCY=${MAKE_CONCURRENCY:-4}

export BEEGFS_VERSION_PATH=`pwd`/../../beegfs-version

set -e

pushd ../ 
TOOLS_ROOT=`pwd`
popd 
BUILD="${TOOLS_ROOT}/build"
mkdir -p $BUILD/buildroot/{BUILD,RPMS,S{RPMS,PECS,OURCES}}

rm -f $BUILD/buildroot/SOURCES/*.tgz

ROOT_DIR=${ROOT_DIR:-`(cd ../..; pwd)`}
BEEGFS_COMMON_PACKAGE_PATH=${BEEGFS_COMMON_PACKAGE_PATH:-`readlink -f ../../beegfs_common_package`}

set +e 
# test if redhat-rpm-config is installed. If so, we must not specify
# %debug_package in the spec file, if not installed we need to define it
# in order to strip binaries and to build a debug package
rpm -qa redhat-rpm-config | grep redhat-rpm-config
NEED_DEBUG_DEFINE=$?

set -e


pushd $TOOLS_ROOT
tar czf $BUILD/buildroot/SOURCES/${NAME}-${BEEGFS_VERSION}.tgz   --exclude=buildroot/* *
cd $BUILD
sed -e "s#%BEEGFS_COMMON_PACKAGE_PATH%#$BEEGFS_COMMON_PACKAGE_PATH#g" \
	-e "s/%NAME%/$NAME/g"					\
	-e "s/%BEEGFS_VERSION%/$BEEGFS_VERSION/g"   					\
	-e "s/%BUILDARCH%/$BUILDARCH/g"   			\
	-e "s/%MAKE_CONCURRENCY%/$CONCURRENCY/g"		\
	-e "s/%NEED_DEBUG_DEFINE%/$NEED_DEBUG_DEFINE/g" \
	${SPEC_FILE}.in > ${SPEC_FILE}

rpmbuild --clean -bb ${SPEC_FILE} --define "_topdir $BUILD/buildroot/" -D "ROOT_DIR $ROOT_DIR"

popd

rm -fr /var/tmp/${NAME}-root/
rm -f /var/tmp/rpm-tmp.*

分析RPM配置

• 檢視SPEC中引用的引數，正式上面的指令碼替換後的結果：

$ cat beegfs_meta/build/beegfs-meta.spec
%define buildarch x86_64
%define BEEGFS_COMMON_PACKAGE_PATH /root/beegfs-v6/fhgfs_common_package
%define BEEGFS_VERSION %VER%-%RELEASE_STR%
%define MAKE_CONCURRENCY 4
%define NEED_DEBUG_DEFINE 0

%define BEEGFS_VERSION 19.4-git17
%define VER %(echo '%{BEEGFS_VERSION}' | cut -d - -f 1)
%define BEEGFS_RELEASE_STR %(echo '%{BEEGFS_VERSION}-' | cut -d - -f 2)
%define EPOCH %(echo '%{VER}' | cut -d . -f 1)

...

%build
cd build
make \
	root-dir=%{ROOT_DIR} \
	BEEGFS_VERSION=%{BEEGFS_VERSION}		\
	-j %{MAKE_CONCURRENCY}
...


$ grep -rHn root-dir beegfs_meta 
beegfs_meta/build/Makefile:8:include $(or $(root-dir),../..)/build/Makefile
beegfs_meta/build/beegfs-meta.spec.in:74:	root-dir=%{ROOT_DIR} \
beegfs_meta/build/beegfs-meta.spec:74:	root-dir=%{ROOT_DIR} \

正式開始編譯

• 先編譯依賴的函式庫：

$ make -j$(nproc) -C beegfs_thirdparty/build
$ make -j$(nproc) -C beegfs_opentk_lib/build
$ make -j$(nproc) -C beegfs_common/build

• 根據需要編譯所需的模組：

$ make -j$(nproc) -C beegfs_meta/build clean
# make -j$(nproc) -C beegfs_meta/build BEEGFS_DEBUG=1
$ make -j$(nproc) -C beegfs_meta/build
[VER]   BEEGFS_VERSION="19.5-git9" BEEGFS_VERSION_CODE=18023689
make: Entering directory `/root/beegfs-v6/fhgfs_meta/build'
[CXX]   ../source/app/App.cpp.o
[CXX]   ../source/components/DatagramListener.cpp.o
[CXX]   ../source/app/config/Config.cpp.o
...
[CXX]   ../source/toolkit/BuddyCommTk.cpp.o
[CXX]   ../source/toolkit/StorageTkEx.cpp.o
[CXX]   ../source/toolkit/XAttrTk.cpp.o
[LD]	beegfs-meta
make: Leaving directory `/root/beegfs-v6/fhgfs_meta/build'

$ file /root/beegfs-v6/beegfs_meta/build/beegfs-meta
/root/beegfs-v6/beegfs_meta/build/beegfs-meta: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=be90bbf04db11b7dcf46d29240b7cb20ab676cf5, not stripped

• 也可以編譯Debug版本，但是要注意，此時前面依賴的函式庫也得編譯成Debug版本，否則執行時會直接崩潰：

$ make -j$(nproc) -C beegfs_meta/build BEEGFS_DEBUG=1
[VER]   BEEGFS_VERSION="19.5-git9" BEEGFS_VERSION_CODE=18023689
make: Entering directory `/root/beegfs-v6/fhgfs_meta/build'
make: Nothing to be done for `all'.
make: Leaving directory `/root/beegfs-v6/fhgfs_meta/build'
[root@sacd02 beegfs-v6]# make -j$(nproc) -C beegfs_meta/build rpm
[VER]   BEEGFS_VERSION="19.5-git9" BEEGFS_VERSION_CODE=18023689
make: Entering directory `/root/beegfs-v6/fhgfs_meta/build'
./make-rpm
~/beegfs-v6/fhgfs_meta ~/beegfs-v6/fhgfs_meta/build
~/beegfs-v6/fhgfs_meta/build
redhat-rpm-config-9.1.0-87.el7.centos.noarch
~/beegfs-v6/fhgfs_meta ~/beegfs-v6/fhgfs_meta/build
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.0aowao
+ umask 022
+ cd /root/beegfs-v6/fhgfs_meta/build/buildroot//BUILD
+ cd /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILD
+ rm -rf beegfs-meta-19.5
+ /usr/bin/mkdir -p beegfs-meta-19.5
+ cd beegfs-meta-19.5
+ /usr/bin/gzip -dc /root/beegfs-v6/fhgfs_meta/build/buildroot/SOURCES/beegfs-meta-19.5-git9.tgz
+ /usr/bin/tar -xvvf -
drwxr-xr-x root/root         0 2019-05-09 10:32 build/
-rw-r--r-- root/root       482 2019-05-08 17:43 build/Makefile
-rw-r--r-- root/root      3994 2019-05-08 17:43 build/beegfs-meta.spec.in
-rw-r--r-- root/root       199 2019-05-08 17:43 build/config.cpack
drwxr-xr-x root/root         0 2019-05-08 17:43 build/debian/
-rw-r--r-- root/root       176 2019-05-08 17:43 build/debian/changelog.in
-rw-r--r-- root/root         2 2019-05-08 17:43 build/debian/compat

...

-rw-r--r-- root/root     27079 2019-05-08 22:43 source/toolkit/BuddyCommTk.cpp.d
-rw-r--r-- root/root     26516 2019-05-08 22:43 source/toolkit/XAttrTk.cpp.d
-rw-r--r-- root/root   2571472 2019-05-08 22:43 source/toolkit/StorageTkEx.cpp.o
-rw-r--r-- root/root   2405768 2019-05-08 22:43 source/toolkit/XAttrTk.cpp.o
-rw-r--r-- root/root   3042288 2019-05-08 22:43 source/toolkit/BuddyCommTk.cpp.o
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ exit 0
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.FjeeXr
+ umask 022
+ cd /root/beegfs-v6/fhgfs_meta/build/buildroot//BUILD
+ cd beegfs-meta-19.5
+ cd build
+ make root-dir=/root/beegfs-v6 BEEGFS_VERSION=19.5-git9 -j 4
[VER]   BEEGFS_VERSION="19.5-git9" BEEGFS_VERSION_CODE=18023689
make[1]: Entering directory `/root/beegfs-v6/fhgfs_meta/build/buildroot/BUILD/beegfs-meta-19.5/build'
make[1]: warning: -jN forced in submake: disabling jobserver mode.
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/root/beegfs-v6/fhgfs_meta/build/buildroot/BUILD/beegfs-meta-19.5/build'
+ exit 0
Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.OeYWrw
+ umask 022
+ cd /root/beegfs-v6/fhgfs_meta/build/buildroot//BUILD
+ '[' /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64 '!=' / ']'
+ rm -rf /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64
++ dirname /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64
+ mkdir -p /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT
+ mkdir /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64
+ cd beegfs-meta-19.5
+ cd build
+ echo 'mkdir RPM_BUILD_ROOT (/root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64)'
mkdir RPM_BUILD_ROOT (/root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64)
+ mkdir -p /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64/etc/beegfs/
+ cp -a dist/etc/beegfs-meta.conf /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64/etc/beegfs/
+ mkdir -p /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64/etc/init.d/
+ INITSCRIPT=dist/etc/init.d/beegfs-meta.init
+ install -D dist/etc/init.d/beegfs-meta.init /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64/etc/init.d/beegfs-meta
+ install -D -m644 dist/usr/lib/systemd/system/beegfs-meta.service /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64/usr/lib/systemd/system/beegfs-meta.service
+ GENERIC_INITSCRIPT=/root/beegfs-v6/fhgfs_common_package/build/dist/etc/init.d/beegfs-service.init
+ cat /root/beegfs-v6/fhgfs_common_package/build/dist/etc/init.d/beegfs-service.init
+ install -D beegfs-meta /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64/opt/beegfs/sbin/beegfs-meta
+ install -D dist/sbin/beegfs-setup-meta /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64/opt/beegfs/sbin/beegfs-setup-meta
+ install -D dist/etc/default/beegfs-meta /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64/etc/default/beegfs-meta
+ /usr/lib/rpm/find-debuginfo.sh --strict-build-id -m --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 /root/beegfs-v6/fhgfs_meta/build/buildroot//BUILD/beegfs-meta-19.5
extracting debug info from /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64/opt/beegfs/sbin/beegfs-meta
dwz: Too few files for multifile optimization
/usr/lib/rpm/sepdebugcrcfix: Updated 1 CRC32s, 0 CRC32s did match.
+ /usr/lib/rpm/check-buildroot
+ /usr/lib/rpm/redhat/brp-compress
+ /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip
+ /usr/lib/rpm/brp-python-bytecompile /usr/bin/python 1
+ /usr/lib/rpm/redhat/brp-python-hardlink
+ /usr/lib/rpm/redhat/brp-java-repack-jars
Processing files: beegfs-meta-19.5-git9el7.x86_64
Provides: beegfs-meta = 19:19.5-git9el7 beegfs-meta(x86-64) = 19:19.5-git9el7 config(beegfs-meta) = 19:19.5-git9el7
Requires(interp): /bin/sh /bin/sh
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires(post): /bin/sh
Requires(preun): /bin/sh
Requires: /bin/bash libbeegfs-opentk.so()(64bit) libc.so.6()(64bit) libc.so.6(GLIBC_2.14)(64bit) libc.so.6(GLIBC_2.2.5)(64bit) libc.so.6(GLIBC_2.3)(64bit) libc.so.6(GLIBC_2.3.2)(64bit) libc.so.6(GLIBC_2.3.3)(64bit) libc.so.6(GLIBC_2.3.4)(64bit) libc.so.6(GLIBC_2.4)(64bit) libcppunit-1.12.so.1()(64bit) libdl.so.2()(64bit) libgcc_s.so.1()(64bit) libgcc_s.so.1(GCC_3.0)(64bit) libm.so.6()(64bit) libm.so.6(GLIBC_2.2.5)(64bit) libpthread.so.0()(64bit) libpthread.so.0(GLIBC_2.2.5)(64bit) libpthread.so.0(GLIBC_2.3.2)(64bit) libpthread.so.0(GLIBC_2.3.3)(64bit) libpthread.so.0(GLIBC_2.3.4)(64bit) librt.so.1()(64bit) librt.so.1(GLIBC_2.2.5)(64bit) libstdc++.so.6()(64bit) libstdc++.so.6(CXXABI_1.3)(64bit) libstdc++.so.6(CXXABI_1.3.5)(64bit) libstdc++.so.6(GLIBCXX_3.4)(64bit) libstdc++.so.6(GLIBCXX_3.4.11)(64bit) libstdc++.so.6(GLIBCXX_3.4.15)(64bit) libstdc++.so.6(GLIBCXX_3.4.19)(64bit) libstdc++.so.6(GLIBCXX_3.4.9)(64bit) rtld(GNU_HASH)
Processing files: beegfs-meta-debuginfo-19.5-git9el7.x86_64
Provides: beegfs-meta-debuginfo = 19:19.5-git9el7 beegfs-meta-debuginfo(x86-64) = 19:19.5-git9el7
Requires(rpmlib): rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(CompressedFileNames) <= 3.0.4-1
Checking for unpackaged file(s): /usr/lib/rpm/check-files /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64
Wrote: /root/beegfs-v6/fhgfs_meta/build/buildroot/RPMS/x86_64/beegfs-meta-19.5-git9el7.x86_64.rpm
Wrote: /root/beegfs-v6/fhgfs_meta/build/buildroot/RPMS/x86_64/beegfs-meta-debuginfo-19.5-git9el7.x86_64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.jbV8Ew
+ umask 022
+ cd /root/beegfs-v6/fhgfs_meta/build/buildroot//BUILD
+ cd beegfs-meta-19.5
+ rm -rf /root/beegfs-v6/fhgfs_meta/build/buildroot/BUILDROOT/beegfs-meta-19.5-git9el7.x86_64
+ exit 0
Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.IPpOum
+ umask 022
+ cd /root/beegfs-v6/fhgfs_meta/build/buildroot//BUILD
+ rm -rf beegfs-meta-19.5
+ exit 0
~/beegfs-v6/fhgfs_meta/build
make: Leaving directory `/root/beegfs-v6/fhgfs_meta/build'

$ ll beegfs_meta/build/buildroot/RPMS/x86_64/
total 18036
-rw-r--r-- 1 root root  1165336 May  9 10:33 beegfs-meta-19.5-git9el7.x86_64.rpm
-rw-r--r-- 1 root root 17300984 May  9 10:33 beegfs-meta-debuginfo-19.5-git9el7.x86_64.rpm

分析啟動指令碼

• BeeGFS使用Systemd進行管理，其配置檔案中直接指定執行程式的路徑和引數：

$ systemctl status beegfs-meta.service 
● beegfs-meta.service - Start BeeGFS Metadata Server
   Loaded: loaded (/usr/lib/systemd/system/beegfs-meta.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2019-05-08 22:33:19 CST; 10min ago
  Process: 48475 ExecStop=/etc/init.d/beegfs-meta stop (code=exited, status=0/SUCCESS)
  Process: 48685 ExecStart=/etc/init.d/beegfs-meta start (code=exited, status=0/SUCCESS)
 Main PID: 48691 (beegfs-meta/Mai)
    Tasks: 442
   Memory: 363.9M
   CGroup: /system.slice/beegfs-meta.service
           └─48691 /opt/beegfs/sbin/beegfs-meta cfgFile=/etc/beegfs/beegfs-meta.conf pidFile=/var/run/beegfs-meta.pid

May 08 22:33:19 sacd02 systemd[1]: Starting Start BeeGFS Metadata Server...
May 08 22:33:19 sacd02 beegfs-meta[48685]: Starting BeeGFS Meta Server: [  OK  ]
May 08 22:33:19 sacd02 systemd[1]: Started Start BeeGFS Metadata Server.

$ cat /usr/lib/systemd/system/beegfs-meta.service
[Unit]
Description=Start BeeGFS Metadata Server
Requires=network-online.target
# We disable the wants service, because it spams the log files
#Wants=beegfs-mgmtd.service beegfs-storage.service openibd.service openib.service rdma.service opensmd.service opensm.service
After=network-online.target beegfs-mgmtd.service beegfs-storage.service openibd.service openib.service rdma.service opensmd.service opensm.service zfs.target

[Service]
ExecStart=/etc/init.d/beegfs-meta start
ExecStop=/etc/init.d/beegfs-meta stop
Type=forking
# We disable PIDFile= because it doesn't work with multi-mode configurations
#PIDFile=/var/run/beegfs-meta.pid

[Install]
WantedBy=multi-user.target

• 為了保持跟早期的SysInitV保持相容，整整的程式執行指令碼在init.d目錄中：

$ cat /etc/init.d/beegfs-meta
#!/bin/bash
#
### BEGIN INIT INFO
# Provides:          beegfs-meta
# Required-Start:
# Should-Start:      $network beegfs-mgmtd beegfs-storage openibd openib rdma opensmd opensm
# Required-Stop:
# Should-Stop:       $network beegfs-mgmtd beegfs-storage openibd openib rdma opensmd opensm
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# chkconfig:         35 95 9
# Short-Description: BeeGFS Meta
# Description:       Start BeeGFS Meta
### END INIT INFO

APP_NAME="BeeGFS Meta Server"
SERVICE_NAME=beegfs-meta

# source function library
. /etc/beegfs/lib/start-stop-functions
. /etc/beegfs/lib/init-multi-mode

SYSCONFIG_FILE=/etc/default/${SERVICE_NAME}

# Check for missing binaries (stale symlinks should not happen)
# Note: Special treatment of stop for LSB conformance
APP_BIN=/opt/beegfs/sbin/${SERVICE_NAME}
test -x $APP_BIN || { echo "$APP_BIN not installed"; 
   if [ "$1" = "stop" ]; then exit 0;
   else exit 5; fi; }

source ${SYSCONFIG_FILE}
if [ "${MULTI_MODE}" = "YES" -o "${MULTI_MODE}" = "yes" ]; then
   init_multi_mode $1 $2
   exit $?
fi

# Check for existence of needed config file and read it
APP_CONFIG=/etc/beegfs/${SERVICE_NAME}.conf
test -r $APP_CONFIG || { echo "$APP_CONFIG not existing";
   if [ "$1" = "stop" ]; then exit 0;
   else exit 6; fi; }

PIDFILE=/var/run/${SERVICE_NAME}.pid
DAEMON_ARGS="cfgFile=${APP_CONFIG} pidFile=$PIDFILE"

RETVAL=0
# Return values acc. to LSB for all commands but status:
# 0       - success
# 1       - generic or unspecified error
# 2       - invalid or excess argument(s)
# 3       - unimplemented feature (e.g. "reload")
# 4       - user had insufficient privileges
# 5       - program is not installed
# 6       - program is not configured
# 7       - program is not running
# 8--199  - reserved (8--99 LSB, 100--149 distrib, 150--199 appl)
# 
# Note that starting an already running service, stopping
# or restarting a not-running service as well as the restart
# with force-reload (in case signaling is not supported) are
# considered a success.

case "$1" in
   start)
      if [ -f "${SYSCONFIG_FILE}" ]; then
         if [ "${START_SERVICE}" = "NO" -o "${START_SERVICE}" = "no" ]; then
            echo "${APP_NAME} not set to be started"
            exit 0
         fi
      fi

      echo -n "Starting ${APP_NAME}: "
      ## create subfolder for lock files, on Debian systems needed
      mkdir -p /var/lock/subsys

      ## Start daemon with startproc(8). If this fails
      ## the return value is set appropriately by startproc.
      daemon --pidfile $PIDFILE $APP_BIN $DAEMON_ARGS \
         && touch /var/lock/subsys/${SERVICE_NAME}
      RETVAL=$?
      echo
      ;;
   stop)
      echo -n "Shutting down ${APP_NAME}: "
      ## Stop daemon with killproc(8) and if this fails
      ## killproc sets the return value according to LSB.

      killproc -p $PIDFILE $APP_BIN && rm -f /var/lock/subsys/${SERVICE_NAME}
      RETVAL=$?
      echo
      ;;
   restart)
      ## Stop the service and regardless of whether it was
      ## running or not, start it again.
      $0 stop
      $0 start
      RETVAL=$?
      ;;
   status)
      echo -n "Checking for service ${APP_NAME}: "
      ## Check status with checkproc(8), if process is running
      ## checkproc will return with exit status 0.

      # Return value is slightly different for the status command:
      # 0 - service up and running
      # 1 - service dead, but /var/run/  pid  file exists
      # 2 - service dead, but /var/lock/ lock file exists
      # 3 - service not running (unused)
      # 4 - service status unknown :-(
      # 5--199 reserved (5--99 LSB, 100--149 distro, 150--199 appl.)

      status -p $PIDFILE $APP_BIN
      RETVAL=$?
      echo
      ;;
   *)
      echo "Usage: $0 {start|stop|status|restart}"
      exit 3
      ;;
   esac
exit $RETVAL

執行編譯程式

• 我們只需要把BeeGFS的SysInitV的啟動指令碼中改成自己編譯的程式，就可以直接無縫運行了：

$ ps -aux | grep beegfs-meta
root      48691 36.7  0.0 33435824 338600 ?     Ssl  22:33   4:09 /opt/beegfs/sbin/beegfs-meta cfgFile=/etc/beegfs/beegfs-meta.conf pidFile=/var/run/beegfs-meta.pid
root      51229  0.0  0.0 112712   980 pts/1    R+   22:44   0:00 grep --color=auto beegfs-meta

$ vi /etc/init.d/beegfs-meta
#APP_BIN=/opt/beegfs/sbin/${SERVICE_NAME}
APP_BIN=/root/beegfs-v6/beegfs_meta/build/${SERVICE_NAME}

# /root/beegfs-v6/beegfs_meta/build/beegfs-meta cfgFile=/etc/beegfs/beegfs-meta.conf pidFile=/var/run/beegfs-meta.pid
$ systemctl stop beegfs-meta.service

$ beegfs-ctl --listtargets --state --nodetype=meta
TargetID     Reachability  Consistency   NodeID
========     ============  ===========   ======
       2           Online         Good        2
       3          Offline Needs-resync        3

$ systemctl start beegfs-meta.service

$ beegfs-ctl --listtargets --state --nodetype=meta
TargetID     Reachability  Consistency   NodeID
========     ============  ===========   ======
       2           Online         Good        2
       3           Online    resyncing        3

$ systemctl status beegfs-meta.service
● beegfs-meta.service - Start BeeGFS Metadata Server
   Loaded: loaded (/usr/lib/systemd/system/beegfs-meta.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2019-05-09 11:01:41 CST; 21s ago
  Process: 60986 ExecStop=/etc/init.d/beegfs-meta stop (code=exited, status=0/SUCCESS)
  Process: 62824 ExecStart=/etc/init.d/beegfs-meta start (code=exited, status=0/SUCCESS)
 Main PID: 62834 (beegfs-meta/Mai)
    Tasks: 441
   Memory: 53.4M
   CGroup: /system.slice/beegfs-meta.service
           └─62834 /root/beegfs-v6/beegfs_meta/build/beegfs-meta cfgFile=/etc/beegfs/beegfs-meta.conf pidFile=/var/run/beegfs-meta.pid

May 09 11:01:40 sacd02 systemd[1]: Starting Start BeeGFS Metadata Server...
May 09 11:01:41 sacd02 beegfs-meta[62824]: Starting BeeGFS Meta Server: [  OK  ]
May 09 11:01:41 sacd02 systemd[1]: Started Start BeeGFS Metadata Server.


$ ps -aux | grep beegfs-meta
root      62834  146  0.0 33364184 52596 ?      Ssl  11:01   6:34 /root/beegfs-v6/beegfs_meta/build/beegfs-meta cfgFile=/etc/beegfs/beegfs-meta.conf pidFile=/var/run/beegfs-meta.pid
root      63289  0.0  0.0 112708   980 pts/2    S+   11:06   0:00 grep --color=auto