Devops關鍵工具及技術(一)—Jenkins 容器化
在進行Devops思想與方法論落地的過程中,Jenkins這個開源的軟體基本上會成為我們的首選,因為它的成熟度以及外掛的豐富程度都無法讓我們拒絕它。而隨著Jenkins 2.0的釋出,Pipeline As Code的理念,無疑給Jenkins使用者在Devops落地過程中更加得心應手。
Devops盛行之下,Docker無疑慢慢成為我們應用部署方式的首選,Docker在效率、隔離型、移植性、複用性相對傳統的虛擬機器部署優勢都很明顯。
Devops工具鏈的容器化在Devops整體體系的更快更簡單地落地都起著至關重要的作用。下面就介紹Jenkins的容器化,包括Jenkins Master容器化、Jenkins Slave 容器化以及使用Jenkins的Docker Cloud 外掛進行Slave的雲部署。以及對於Jenkins容器化後內建構建環境安裝以及軟體環境,包括maven、ruby、python、golang、ruby、php、dotnet、git、docker、nodejs、ant等等。
1、Jenkins Master
Jenkins的Master映象構建。首先如果我們需要用我們的Jenkins來作為Devops持續構建和持續部署的工具,那對應的構建不同語言應用的環境那也是必須要構建到我們Master的映象中。下面就是Jenkins Master的Dockerfile部分檔案內容(由於檔案內容過長,只黏貼一部分,所有內容可以檢視上面筆者的Github地址)。
... ... # Install NodeJs 10.x , if you don't need, you can remove it,you also can change version RUN curl -sL https://deb.nodesource.com/setup_10.x | bash - RUN apt-get update && apt-get install -y \ nodejs # Install Python3.7 , if you don't need, you can remove it ,you also can change version RUN wget https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tgz RUN tar xvf Python-3.7.0.tgz RUN rm -rf Python-3.7.0.tgz RUN cd Python-3.7.0 \ ./configure --enable-optimizations \ make -j8 \ make altinstall # Install Golang1.10.3 , if you don't need, you can remove it,you also can change version ENV GOLANG_VERSION 1.10.3 RUN set -eux; \ \ # this "case" statement is generated via "update.sh" dpkgArch="$(dpkg --print-architecture)"; \ case "${dpkgArch##*-}" in \ amd64) goRelArch='linux-amd64'; goRelSha256='fa1b0e45d3b647c252f51f5e1204aba049cde4af177ef9f2181f43004f901035' ;; \ armhf) goRelArch='linux-armv6l'; goRelSha256='d3df3fa3d153e81041af24f31a82f86a21cb7b92c1b5552fb621bad0320f06b6' ;; \ arm64) goRelArch='linux-arm64'; goRelSha256='355128a05b456c9e68792143801ad18e0431510a53857f640f7b30ba92624ed2' ;; \ i386) goRelArch='linux-386'; goRelSha256='3d5fe1932c904a01acb13dae07a5835bffafef38bef9e5a05450c52948ebdeb4' ;; \ ppc64el) goRelArch='linux-ppc64le'; goRelSha256='f3640b2f0990a9617c937775f669ee18f10a82e424e5f87a8ce794a6407b8347' ;; \ s390x) goRelArch='linux-s390x'; goRelSha256='34385f64651f82fbc11dc43bdc410c2abda237bdef87f3a430d35a508ec3ce0d' ;; \ *) goRelArch='src'; goRelSha256='567b1cc66c9704d1c019c50bef946272e911ec6baf244310f87f4e678be155f2'; \ echo >&2; echo >&2 "warning: current architecture ($dpkgArch) does not have a corresponding Go binary release; will be building from source"; echo >&2 ;; \ esac; \ \ url="https://golang.org/dl/go${GOLANG_VERSION}.${goRelArch}.tar.gz"; \ wget -O go.tgz "$url"; \ echo "${goRelSha256} *go.tgz" | sha256sum -c -; \ tar -C /usr/local -xzf go.tgz; \ rm go.tgz; \ \ if [ "$goRelArch" = 'src' ]; then \ echo >&2; \ echo >&2 'error: UNIMPLEMENTED'; \ echo >&2 'TODO install golang-any from jessie-backports for GOROOT_BOOTSTRAP (and uninstall after build)'; \ echo >&2; \ exit 1; \ fi; \ \ export PATH="/usr/local/go/bin:$PATH"; \ go version ENV GOPATH /go ENV PATH $GOPATH/bin:/usr/local/go/bin:$PATH RUN mkdir -p "$GOPATH/src" "$GOPATH/bin" && chmod -R 777 "$GOPATH" WORKDIR $GOPATH # Install Dotnet-SDK 2.1.302, if you don't need, you can remove it,you also can change version RUN wget https://download.microsoft.com/download/4/0/9/40920432-3302-47a8-b13c-bbc4848ad114/dotnet-sdk-2.1.302-linux-x64.tar.gz RUN mkdir -p /home/dotnet && tar zxf dotnet-sdk-2.1.302-linux-x64.tar.gz -C /home/dotnet ENV DOTNET_ROOT $PATH:/home/dotnet ENV PATH $PATH:/home/dotnet RUN rm -rf dotnet-sdk-2.1.302-linux-x64.tar.gz # Install dockerce for build docker images in docker for jenkins, if you don't need, you can remove it,you also can change version RUN apt-get update && \ apt-get -y install apt-transport-https \ ca-certificates \ curl \ gnupg2 \ software-properties-common && \ curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg > /tmp/dkey; apt-key add /tmp/dkey && \ add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \ $(lsb_release -cs) \ stable" && \ apt-get update && \ apt-get -y install docker-ce RUN apt-get clean && apt-get autoclean # Install Jenkins master ARG user=jenkins ARG group=jenkins ARG uid=1000 ARG gid=1000 ARG http_port=8080 ARG agent_port=50000 ARG JENKINS_HOME=/var/jenkins_home ARG bbdocker=12345 ENV JENKINS_HOME $JENKINS_HOME ENV JENKINS_SLAVE_AGENT_PORT ${agent_port} # Jenkins is run with user `jenkins`, uid = 1000 # If you bind mount a volume from the host or a data container, # ensure you use the same uid RUN mkdir -p $JENKINS_HOME \ && chown ${uid}:${gid} $JENKINS_HOME \ && groupadd -g ${gid} ${group} \ && useradd -d "$JENKINS_HOME" -u ${uid} -g ${gid} -m -s /bin/bash ${user} RUN groupadd -g ${bbdocker} bbdocker && gpasswd -a ${user} bbdocker # Jenkins home directory is a volume, so configuration and build history # can be persisted and survive image upgrades VOLUME $JENKINS_HOME # `/usr/share/jenkins/ref/` contains all reference configuration we want # to set on a fresh new installation. Use it to bundle additional plugins # or config file with your custom jenkins Docker image. RUN mkdir -p /usr/share/jenkins/ref/init.groovy.d ... ...
Dockerfile 中其他需要包含應用的檔案可以直接在github中找到,如果有特殊的一些執行環境以及構建環境直接在Dockerfile裡面新增即可。
之後可按照以下步驟進行Jenkins Master的構建與部署。
- 映象構建
docker build -t jenkins:master .
- 執行配置化檔案
./jenkins_dockerconfig.sh
- 啟動Jenkins的Master
./jenkins_master_run.sh
- 訪問vm ip+10000。開始配置Jenkins Master
密碼我們可以通過三種方式獲取
1、訪問Jenkins Master宿主機上的目錄檔案 /opt/jenkins_home/secrets/initialAdminPassword 2、直接檢視容器日誌 docker logs -f dockerid 3、到容器內部檢視。推薦以上兩種
-
等待自動Jenkins外掛安裝
-
建立一個管理員
-
建立Master Jenkins中軟體版本的pipeline並執行
pipeline code可在github中檢視
2、Jenkins Slave
Jenkins的Slave映象構建。Master映象對應構建不同語言應用的環境那也是必須要構建到我們Slave的映象中,因為一旦構建程序增多以後,構建的將會由各個Slave Jenkins去承擔,所有Slave節點必須要有構建不同語言的能力。下面就是Jenkins Slave的Dockerfile部分檔案內容(由於檔案內容過長,只黏貼一部分,所有內容可以檢視上面筆者的Github地址)。
...
...
# Install NodeJs 10.x , if you don't need, you can remove it,you also can change version
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash -
RUN apt-get install nodejs
# Install Python3.7 , if you don't need, you can remove it ,you also can change version
RUN wget https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tgz
RUN tar xvf Python-3.7.0.tgz
RUN rm -rf Python-3.7.0.tgz
RUN cd Python-3.7.0 \
./configure --enable-optimizations \
make -j8 \
make altinstall
# Install Golang1.10.3 , if you don't need, you can remove it,you also can change version
ENV GOLANG_VERSION 1.10.3
RUN set -eux; \
\
# this "case" statement is generated via "update.sh"
dpkgArch="$(dpkg --print-architecture)"; \
case "${dpkgArch##*-}" in \
amd64) goRelArch='linux-amd64'; goRelSha256='fa1b0e45d3b647c252f51f5e1204aba049cde4af177ef9f2181f43004f901035' ;; \
armhf) goRelArch='linux-armv6l'; goRelSha256='d3df3fa3d153e81041af24f31a82f86a21cb7b92c1b5552fb621bad0320f06b6' ;; \
arm64) goRelArch='linux-arm64'; goRelSha256='355128a05b456c9e68792143801ad18e0431510a53857f640f7b30ba92624ed2' ;; \
i386) goRelArch='linux-386'; goRelSha256='3d5fe1932c904a01acb13dae07a5835bffafef38bef9e5a05450c52948ebdeb4' ;; \
ppc64el) goRelArch='linux-ppc64le'; goRelSha256='f3640b2f0990a9617c937775f669ee18f10a82e424e5f87a8ce794a6407b8347' ;; \
s390x) goRelArch='linux-s390x'; goRelSha256='34385f64651f82fbc11dc43bdc410c2abda237bdef87f3a430d35a508ec3ce0d' ;; \
*) goRelArch='src'; goRelSha256='567b1cc66c9704d1c019c50bef946272e911ec6baf244310f87f4e678be155f2'; \
echo >&2; echo >&2 "warning: current architecture ($dpkgArch) does not have a corresponding Go binary release; will be building from source"; echo >&2 ;; \
esac; \
\
url="https://golang.org/dl/go${GOLANG_VERSION}.${goRelArch}.tar.gz"; \
wget -O go.tgz "$url"; \
echo "${goRelSha256} *go.tgz" | sha256sum -c -; \
tar -C /usr/local -xzf go.tgz; \
rm go.tgz; \
\
if [ "$goRelArch" = 'src' ]; then \
echo >&2; \
echo >&2 'error: UNIMPLEMENTED'; \
echo >&2 'TODO install golang-any from jessie-backports for GOROOT_BOOTSTRAP (and uninstall after build)'; \
echo >&2; \
exit 1; \
fi; \
\
export PATH="/usr/local/go/bin:$PATH"; \
go version
ENV GOPATH /go
ENV PATH $GOPATH/bin:/usr/local/go/bin:$PATH
RUN mkdir -p "$GOPATH/src" "$GOPATH/bin" && chmod -R 777 "$GOPATH"
WORKDIR $GOPATH
# Install Dotnet-SDK 2.1.302, if you don't need, you can remove it,you also can change version
RUN wget https://download.microsoft.com/download/4/0/9/40920432-3302-47a8-b13c-bbc4848ad114/dotnet-sdk-2.1.302-linux-x64.tar.gz
RUN mkdir -p /home/dotnet && tar zxf dotnet-sdk-2.1.302-linux-x64.tar.gz -C /home/dotnet
ENV DOTNET_ROOT $PATH:/home/dotnet
ENV PATH $PATH:/home/dotnet
RUN rm -rf dotnet-sdk-2.1.302-linux-x64.tar.gz
# Install dockerce for build docker images in docker for jenkins, if you don't need, you can remove it,you also can change version
RUN apt-get update && \
apt-get -y install apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common && \
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg > /tmp/dkey; apt-key add /tmp/dkey && \
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \
$(lsb_release -cs) \
stable" && \
apt-get update && \
apt-get -y install docker-ce
RUN apt-get clean && apt-get autoclean
# Install Jenkins
ARG user=jenkins
ARG group=jenkins
ARG uid=1000
ARG gid=1000
ARG bbdocker=12345
USER root
ENV HOME /home/${user}
RUN groupadd -g ${gid} ${group}
RUN useradd -c "Jenkins user" -d $HOME -u ${uid} -g ${gid} -m ${user}
LABEL Description="This is a base image, which provides the Jenkins agent executable (slave.jar)" Vendor="Jenkins project" Version="3.20"
RUN groupadd -g ${bbdocker} bbdocker && gpasswd -a ${user} bbdocker
ARG VERSION=3.20
ARG AGENT_WORKDIR=/home/${user}/agent
RUN curl --create-dirs -sSLo /usr/share/jenkins/slave.jar https://repo.jenkins-ci.org/public/org/jenkins-ci/main/remoting/${VERSION}/remoting-${VERSION}.jar \
&& chmod 755 /usr/share/jenkins \
&& chmod 644 /usr/share/jenkins/slave.jar
USER ${user}
ENV AGENT_WORKDIR=${AGENT_WORKDIR}
RUN mkdir /home/${user}/.jenkins && mkdir -p ${AGENT_WORKDIR}
VOLUME /home/${user}/.jenkins
VOLUME ${AGENT_WORKDIR}
WORKDIR /home/${user}
...
...
如果有特殊的一些執行環境以及構建環境直接在Dockerfile裡面新增即可。
之後可按照以下步驟進行Jenkins Slave的構建與部署。
- 映象構建
docker build -t jenkins:slave .
- 更新Slave節點Docker的配置
該配置根據不同的系統可能在/etc/init/docker.conf、/etc/sysconfig/docker、 ,/etc/default/docker 或 /etc/default/docker.io中。
這個配置修改由於Jenkins的Master在啟動Slave進行構件任務的時候是需要開啟Docker的TCP埠。參考官方:https://plugins.jenkins.io/docker-plugin
DOCKER_OPTS="-H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock"
- 重啟Docker
$ systemctl restart docker
- 執行配置化檔案
./jenkins_dockerconfig.sh
- JenkinsMaster節點上配置Slave的資訊
系統管理中的最下面,可以看到Cloud的配置即可以新增Docker,每一個配置為一個Slave的配置。
- 在Master Jenkins中建立Slave Jenkins構建軟體版本的pipeline並執行
pipeline code可在github中檢視
- 構建中可以看到Slave節點上啟動Jenkins Slave進行構建的Docker程序
以上即是Jenkins Master和Slave容器化步驟。我們可以在一臺Jenkins Master中配置0-n臺Slave。而且這些Slave節點將會在需要執行構建任務時才會啟動Jenkins程序進行任務執行,在任務執行完畢後將程序刪除掉,實現了資源的合理利用。而且Slave配置可以進行一些自定義的配置,如Slave節點最高的容器啟動數為多少。這樣可以合理的利用每一臺Slave節點。
相比於傳統部署Jenkins和配置Slave的方案,容器化後的方案在Devops中運用,部署效率和運維效率上都會有很大程度的提示,而且也不依賴與任何基礎環境,不管在Windows下還是Linux下,又或者公有云、私有云下都可以快速地搭建起來。
對於後續不論Jenkins版本升級還是構建環境版本升級只需要修改Dockerfile檔案,重新進行Master和Slave映象的打包即可,這樣,所有事情都變得簡單起來了。