Jenkis pipeline構建項目實踐
阿新 • • 發佈：2018-08-14
gen ica imp maven項目 worker -o ssm tmp ole
在完成前文的jenkins server 在k8s環境部署之後，本文我們來測試在k8s集群環境中的jenkins pipeline構建項目和更新，具體環境要求如下：
1、jenkins pipeline插件安裝成功
2、要更新的應用已提前部署
3、Jenkins slave中需要有kubectl、svn、mvn客戶端且環境變量設置準確
4、Jenkis slave需要能和master的api-server進行正常通信（這裏為了簡便，前文構建jenkins server的時候直接對defalut這個service account做rabc授權。其他可選方案是使用~/.kube/config文件實現客戶端的授權認證）
一、創建構建任務
1、點擊“jenkins ”——“新建 ”——“pipeline”，輸入項目的名稱，點擊“OK”
2、下拉到“pipeline”配置，填寫腳本
具體腳本內容如下：
podTemplate(name: ‘jenkins-slave‘, cloud: ‘kubernetes‘, namespace: ‘default‘, label: ‘jenkins-slave‘, serviceAccount: ‘default‘, containers: [ containerTemplate( name: ‘jenkins-slave‘, image: ‘harbor.59iedu.com/fjhb/jenkins-slave-toolkit:2018-08-10-v1‘, args: ‘${computer.jnlpmac} ${computer.name}‘, ttyEnabled: true, privileged: false, alwaysPullImage:true, ) ], volumes: [ persistentVolumeClaim(mountPath: ‘/tmp/‘, claimName: ‘tomcat-jcsj-data‘) ]) { node(‘jenkins-slave‘) { stage(‘svn-checkout‘) { container(‘jnlp‘) { sh """ svn checkout --username=yanglw --password=mypassword http://192.168.1.81/svn/fjhbjsb/k8s-pipeline-test --non-interactive """ } } stage(‘mvn-package‘) { container(‘jnlp‘) { sh """ mkdir -p /var/jenkins_home && cd k8s-pipeline-test && mvn clean package && cp -rpf target/*.war /tmp/ """ } } stage(‘restart‘) { container(‘jnlp‘) { sh """ pod_name=`kubectl get pods -l name=jcsj-dev -o name | cut -d"/" -f2` kubectl delete pod \$pod_name """ } } } }
3、腳本釋義
podTemplate節指定創建pod的模板和環境
Name：為pod的名稱前綴
Cloud：為構建pod的雲環境，需要和前面新建的雲環境名稱一樣
Namespace：創建pod所在的namespace
Label: 創建pod對應的標簽
serviceAccount：pod使用sa，這裏使用default, 所以前文創建jenkins master deployment的時候多創建了一個default-role,這樣自動創建出來的的jenkins-slave具有對應的api權限
containerTemplate節
指定創建容器的模板，當雲環境裏面配置pod模板後，容器模板以雲環境的配置為準。
Volumes節
配置jenkins-slave pod掛載的卷，當雲環境裏面配置pod模板後，掛載卷以雲環境配置為準
stage節為具體的pipeline步驟
這裏第一步進行svn代碼遷出；
第二步進行編譯，並把包傳到共享卷上面；
第三步對pod進行刪除，因為應用采用deployment方式部署，所以可以實現重新創建pod，達到更新的效果。
二、構建準備
在開始構建之前，有必要介紹一下jcsj-dev的環境，這個環境是需要提前部署好的。部署文件如下
# cat deploy.yml
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: jcsj-dev
namespace: default
spec:
replicas: 1
template:
metadata:
labels:
name: jcsj-dev
spec:
containers:
- name: jcsj-dev
image: tomcat:latest
imagePullPolicy: IfNotPresent
ports:
- name: web
containerPort: 8080
volumeMounts:
- mountPath: /usr/local/tomcat/webapps
name: tomcat-jcsj-data
volumes:
- name: tomcat-jcsj-data
persistentVolumeClaim:
claimName: tomcat-jcsj-data
---
apiVersion: v1
kind: Service
metadata:
name: jcsj-dev
spec:
type: NodePort
ports:
- port: 8080
targetPort: 8080
nodePort: 8453
selector:
name: jcsj-dev# cat pv.yaml
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: tomcat-jcsj-data
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteMany
nfs:
path: /home/tomcat_jcsj
server: 192.168.115.6
persistentVolumeReclaimPolicy: Recycle
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: tomcat-jcsj-data
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5Gi
三、更新項目代碼
更新index.jsp並提交
四、開始構建
點擊“立即構建”，構建過程中可以通過控制臺輸出看具體的日誌信息
構建過程中觀察jenkins master的日誌輸出：
# kubectl logs -f jenkins-master-588b89c75f-ztvgm
INFO: Started provisioning Kubernetes Pod Template from kubernetes with 1 executors. Remaining excess workload: 0
Aug 13, 2018 10:14:22 AM hudson.slaves.NodeProvisioner$2 run
INFO: Kubernetes Pod Template provisioning successfully completed. We have now 2 computer(s)
Aug 13, 2018 10:14:22 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Created Pod: jenkins-slave-gjjbh in namespace default
Aug 13, 2018 10:14:22 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for Pod to be scheduled (0/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:28 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Container is waiting jenkins-slave-gjjbh [jnlp]: ContainerStateWaiting(message=null, reason=ContainerCreating, additionalProperties={})
Aug 13, 2018 10:14:28 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for Pod to be scheduled (1/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:34 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (1/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:35 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (2/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:36 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (3/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:37 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (4/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:38 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (5/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:39 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (6/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:40 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (7/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:41 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (8/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:42 AM hudson.TcpSlaveAgentListener$ConnectionHandler run
INFO: Accepted JNLP4-connect connection #1 from /172.30.66.4:47408
Aug 13, 2018 10:14:42 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (9/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:43 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (10/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:44 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (11/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:45 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (12/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:46 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (13/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:47 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (14/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:48 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (15/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:49 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (16/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:50 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (17/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:51 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (18/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:52 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (19/100): jenkins-slave-gjjbh
Aug 13, 2018 10:14:53 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Waiting for agent to connect (20/100): jenkins-slave-gjjbh
Aug 13, 2018 10:15:12 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1 doLaunch
INFO: Created process inside pod: [jenkins-slave-gjjbh], container: [jnlp] with pid:[-1]
Aug 13, 2018 10:15:21 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1 doLaunch
INFO: Created process inside pod: [jenkins-slave-gjjbh], container: [jnlp] with pid:[-1]
Aug 13, 2018 10:16:04 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1 doLaunch
INFO: Created process inside pod: [jenkins-slave-gjjbh], container: [jnlp] with pid:[-1]
Aug 13, 2018 10:16:06 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminating Kubernetes instance for agent jenkins-slave-gjjbh
Aug 13, 2018 10:16:06 AM jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed
WARNING: Computer.threadPoolForRemoting [#8] for jenkins-slave-gjjbh terminated
java.nio.channels.ClosedChannelException
at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:208)
at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:800)
at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173)
at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:311)
at hudson.remoting.Channel.close(Channel.java:1295)
at hudson.remoting.Channel.close(Channel.java:1263)
at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:708)
at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:96)
at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:626)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Aug 13, 2018 10:16:06 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.PodTemplateStepExecution$PodTemplateCallback finished
INFO: Removing pod template and deleting pod jenkins-slave-7gnwx from cloud kubernetes
Aug 13, 2018 10:16:06 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminated Kubernetes instance for agent default/jenkins-slave-gjjbh
Aug 13, 2018 10:16:06 AM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Disconnected computer jenkins-slave-gjjbh
Terminated Kubernetes instance for agent default/jenkins-slave-gjjbh
Aug 13, 2018 10:16:06 AM org.csanchez.jenkins.plugins.kubernetes.pipeline.PodTemplateStepExecution$PodTemplateCallback finished
WARNING: Failed to delete pod for agent default/jenkins-slave-7gnwx: not found
Aug 13, 2018 10:16:07 AM org.jenkinsci.plugins.workflow.job.WorkflowRun finish
INFO: k8s-pipeline-test #1 completed: SUCCESS五、驗證結果
六、後記
本文僅介紹的pipeline方式構建項目的過程，實際應用中還需要解決若幹問題，例如：
1、svn checkout步驟需要明文配置用戶名和密碼，存在安全性問題
2、 每次執行構建都需要全量遷出代碼，效率不高
3、要更新的應用需要提前部署好，不能適配首次編譯部署場景
4、對共享卷存在依賴，解耦問題需要解決
後續將介紹maven項目的構建和自動部署！
Jenkis pipeline構建項目實踐