[k8s] jenkins配合kubernetes外掛實現k8s叢集構建的持續整合
程式設計師將程式碼提交到程式碼倉庫gitlab
- 鉤子觸發jenkins master啟動一次構建
- jenkins master從k8s申請一個jenkins slave編譯容器
- 在容器內編譯完成以後，獲得最終產物
- 將最終產物通過dockerfile生成生產部署映象（這裡省略了測試，其實部署映象需要測試通過）
- 將生產映象推送到harbor映象倉庫
- jenkins slave生命週期結束，k8s銷燬slave容器
- 一次構建完成k8s持續整合的一個思路:
這裡要說的是部署部分
注: 這只是一個持續整合思想.本篇按照這個思想來搞,在我的環境裡我為了速度快當然還有別的因素我用這種方案,用的很6.
其他思路:
- 可以將code+image打在一起做升級
- 可以rbac+環境+ns+supervisor 每個開發一個環境這樣搞
後面我抽空一一實現下.
這篇文章思路:
手動構建war包(整合測試)-->本地tomcat測試通過(功能測試)-->k8s容器化tomcat(pv+deploy+svc+ingress)-->將war包拖入k8s的tomcat測試.
jenkins jnlp映象構建(mvn+git+kubectl)-> jnlp映象測試,確保可被server動態排程-->配置war包的pipeline測試.
注: 本篇jenkins server部署在vm上,非docker部署,jenkins-jnlp-slave是容器化自動建立的.
其他內容參考: 容器ci索引:
構建jnlp映象的dockerfile
- 準備dockerfile所需檔案
git clone https://github.com/jenkinsci/docker-jnlp-slave.git
cd docker-jnlp-slave
$ ls
Dockerfile jenkins-slave kubectl README.md
構建mvn3.5.2+git+kubectl的映象
基於jenkinsci/slave:alpine的基礎映象
參考: https://github.com/jenkinsci/docker-slave/blob/master/Dockerfile
https://github.com/jenkinsci/docker-jnlp-slave/blob/master/Dockerfile
https://hub.docker.com/r/jenkinsci/slave/tags/
alpine-git安裝參考:
https://hub.docker.com/r/alpine/git/~/dockerfile/
$ cat Dockerfile
FROM jenkinsci/slave:alpine
USER root
RUN apk add --no-cache curl tar bash
## Install Maven
ARG MAVEN_VERSION=3.5.2
ARG USER_HOME_DIR="/root"
ARG SHA=707b1f6e390a65bde4af4cdaf2a24d45fc19a6ded00fff02e91626e3e42ceaff
ARG BASE_URL=https://apache.osuosl.org/maven/maven-3/${MAVEN_VERSION}/binaries
RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
&& curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL}/apache-maven-${MAVEN_VERSION}-bin.tar.gz \
&& echo "${SHA} /tmp/apache-maven.tar.gz" | sha256sum -c - \
&& tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
&& rm -f /tmp/apache-maven.tar.gz \
&& ln -s /usr/share/maven/bin/mvn /usr/bin/mvn \
&& apk --update add git openssh \
&& rm -rf /var/lib/apt/lists/* \
&& rm /var/cache/apk/* \
&& mkdir /src /target \
&& chown jenkins.jenkins /src /target
ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
# install kubectl
COPY kubectl /usr/local/bin/kubectl
## install jenkins-slave
COPY jenkins-slave /usr/local/bin/jenkins-slave
USER jenkins
WORKDIR /home/jenkins
ENTRYPOINT ["jenkins-slave"]映象已可以從dockerhub下載:
docker pull lanny/mvn-git-kubectl-jnlp:3.5.2測試jnlp映象
主要看他能否用jenkins-server動態呼叫跑起來
當然首先安裝jenkins kubernetes外掛,新建一朵雲:
參考: http://www.cnblogs.com/iiiiher/p/7979336.html
配置專案: 選擇pipeline script
podTemplate(name: 'maotai-dev', cloud: 'kubernetes',
namespace: 'kube-public', label: 'maotai-dev',
serviceAccount: 'default', containers: [
containerTemplate(
name: 'jnlp',
image: 'lanny/mvn-git-kubectl-jnlp:3.5.2',
args: '${computer.jnlpmac} ${computer.name}',
ttyEnabled: true,
privileged: false,
alwaysPullImage: false)
],
) {
node('maotai-dev') {
stage('git-clone') {
container('jnlp') {
sh """
date +%F;
sleep 30;
"""
}
}
}
}
點選構建-->顯示構建成功
構建成功後jnlp映象隨著構建結束自動刪除.
tomcat java-helloworld專案
kubernetes外掛的pipeline使用:
參考:
https://github.com/jenkinsci/kubernetes-plugin
https://help.aliyun.com/document_detail/56336.html?spm=5176.doc56336.6.851.wAqCzu
javahelloworld程式碼: https://github.com/lannyMa/trucks ,構建可形成helloworld的war包.可以部署在tomcat用於測試.
jenkins專案配置: 新建專案 test-pipeline
podTemplate(name: 'maotai-dev', cloud: 'kubernetes',
namespace: 'kube-public', label: 'maotai-dev',
serviceAccount: 'default', containers: [
containerTemplate(
name: 'jnlp',
image: 'lanny/mvn-git-kubectl-jnlp:3.5.2',
args: '${computer.jnlpmac} ${computer.name}',
ttyEnabled: true,
privileged: false,
alwaysPullImage: false)
],
volumes: [
persistentVolumeClaim(mountPath: '/tmp/', claimName: 'spring-pvc')
]) {
node('maotai-dev') {
stage('git-clone') {
container('jnlp') {
sh """
git clone https://github.com/lannyMa/trucks.git
"""
}
}
stage('mvn-package') {
container('jnlp') {
sh """
cd trucks && mvn clean package && cp -rpf target/*.war /tmp/
"""
}
}
stage('restart') {
container('jnlp') {
sh """
pod_name=`kubectl -s 192.168.x.x:8080 -n kube-public get pods -l name=maotai-dev -o name | cut -d"/" -f2`
kubectl -s kube-apiserver-http.kube-public -n kube-public delete pod \$pod_name
"""
}
}
}
}
配置tomcat專案
tomcat-pvc.yaml #前提是配置好stroragecalss: 參考: http://www.cnblogs.com/iiiiher/p/7988803.html
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: spring-pvc
namespace: kube-public
spec:
storageClassName: "managed-nfs-storage"
accessModes:
- ReadOnlyMany
resources:
requests:
storage: 100Mitomcat-deploy.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: spring
namespace: kube-public
spec:
replicas: 1
template:
metadata:
labels:
name: spring
name: maotai-dev #這裡標籤設定需注意,因為jenkins配置kubectl的stage時需要根據標籤過濾重啟它: kubectl -s 192.168.x.x:8080 -n kube-public get pods -l name=spring -o name | cut -d"/" -f2
spec:
containers:
- name: spring
image: tomcat:latest
imagePullPolicy: IfNotPresent
ports:
- name: web
containerPort: 8080
volumeMounts:
- mountPath: /usr/local/tomcat/webapps
name: spring-folder
volumes:
- name: spring-folder
persistentVolumeClaim:
claimName: spring-pvc執行成功:
k8s叢集容器化tomcat專案
- 容器化tomcat專案: 配置k8s叢集的tomcat 包含了 pvc+deploy+svc+ingress
- 做法:
- 先手動編譯專案,本次tomcat測試通過
- 整合到k8s叢集的tomcat,測試,確保專案可以正常執行
tomcat-svc.yaml
apiVersion: v1
kind: Service
metadata:
name: spring
namespace: kube-public
labels:
name: spring
spec:
ports:
- name: web
port: 8080
targetPort: web
selector:
name: spring
tomcat-ingress.yaml #前提是已配置好了ingress,nginx-ingress配置參考:http://www.cnblogs.com/iiiiher/p/8006801.html
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: spring
namespace: kube-public
spec:
rules:
- host: spring.maotai.net
http:
paths:
- path: /
backend:
serviceName: spring
servicePort: web
建立好後確保能夠訪問:
接下來需要手動編譯,確保專案通過整合測試(可成功編譯),功能測試(部署tomcat後可訪問)
專案程式碼: https://github.com/lannyMa/trucks.git
mvn配置改源等參考: https://github.com/lannyMa/java-helloword.git
確保沒問題後將war包放到上一步建立的pv裡.我的是nfs,直接到nfs-server上把war包託上去,然後重啟tomcat,測試效果.