Devops關鍵工具及技術(八)—基於Pipeline的Ansible自動化部署[二](Ansible自動化部署的流水線整合)
阿新 • • 發佈：2018-12-17
上篇文章中，我們將Ansible進行了容器化。在這篇文章中我們將利用Ansible映象整合到Jenkins的Docker Cloud中，這個過程與之前的Jenkins Slave整合、RobotFramework自動化測試整合、Jmeter效能測試整合一樣。
1、自動化測試節點
我們可以按照Slave、Robot、Jmeter節點方式來將Ansible自動化部署節點加入到Jenkins中，目的為了將Ansible自動化部署成為流水線的一部分。 在此之前我們需要將Ansible容器化，具體可以參考Devops關鍵工具及技術(八)—基於Pipeline的Ansible自動化部署(Ansible容器化)
- JenkinsMaster上配置Ansible自動化部署節點的資訊
在做配置之前我們也還是需要對該加入進來的節點做一下配置，具體的操作可以從上面提到的文章中獲知，這裡不做過多講解。可參考下面的截圖
2、Pipeline流水線的整合
基於之前Jmeter效能測試的流水線，我們加入Ansible自動化部署的Stage。在此之前我們的Pipeline裡面加入了Checkout Code、Mvn Build、Sonar、Bash Deploy、RobotFramework、Jmeter等Stage，這次我們在後面加上Ansible的Stage。
- Ansible自動化部署指令碼
hosts檔案
[all]
192.168.88.128 ansible_ssh_user=root ansible_connection=ssh [email protected]
deploy.yaml檔案
- name: deploy demo jar
hosts: all
gather_facts: no
remote_user: root
roles:
- demo
roles/demo/tasks/main.yaml檔案
- name: Copy sample.jar to remote vm //拷貝檔案 copy: src=/opt/tmp/sample.jar dest=~/ mode=0755 - name: Get running sample.jar //得到原程序id shell: "ps -ef | grep -v grep | grep -w sample.jar | awk '{print $2}'" register: running_processes - name: Kill running sample.jar //殺死原程序 shell: "kill {{ item }}" with_items: "{{ running_processes.stdout_lines }}" - wait_for: //確認程序是否已刪除 path: "/proc/{{ item }}/status" state: absent with_items: "{{ running_processes.stdout_lines }}" ignore_errors: yes register: killed_processes - name: Force kill stuck sample.jar //強制刪除程序 shell: "kill -9 {{ item }}" with_items: "{{ killed_processes.results | select('failed') | map(attribute='item') | list }}" - name: Sleep 10s //睡眠10s shell: sleep 10s; - name: Deploy sample.jar //部署新工程 shell: nohup java -jar ~/sample.jar > ~/nohup.out &
- Pipeline內容
pipeline {
agent none
stages {
stage('Preparation') {
agent { node { label 'master' } }
steps {
checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'binbin', url: 'https://github.com/zbbkeepgoing/springboot-demo.git']]])
}
}
stage('Build') {
agent { node { label 'master' } }
steps {
dir(env.WORKSPACE){
sh "mvn clean install"
junit allowEmptyResults: true, keepLongStdio: true, testResults: 'target/surefire-reports/*.xml'
sh "mv target/sample-0.0.1-SNAPSHOT.jar target/sample.jar"
}
}
}
stage('Sonarqube') {
agent { node { label 'master' } }
steps {
dir(env.WORKSPACE){
sh "mvn sonar:sonar -Dsonar.host.url=http://192.168.88.130:9000 -Dsonar.login=65607ba9d0f54590cf55fe8e60134fb5e87c557d"
}
}
}
stage('Deploy') {
agent { node { label 'master' } }
steps {
dir(env.WORKSPACE){
sh 'sshpass -p [email protected] scp -o StrictHostKeychecking=no target/sample.jar [email protected]:/opt/ansible'
sh 'sshpass -p [email protected] scp -o StrictHostKeychecking=no deploy.sh [email protected]:/opt/ansible'
sh 'sshpass -p [email protected] ssh -o StrictHostKeychecking=no [email protected] "bash /opt/ansible/deploy.sh"'
sh 'sleep 8s'
}
}
}
stage('Robot Framework') {
agent { node { label 'robot' } }
steps {
dir(env.WORKSPACE){
checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'binbin', url: 'https://github.com/zbbkeepgoing/springboot-demo.git']]])
sh "pybot -d /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible robot/demo.robot"
step([$class: 'RobotPublisher',
disableArchiveOutput: false,
logFileName: 'log.html',
otherFiles: '',
outputFileName: 'output.xml',
outputPath: '.',
passThreshold: 40,
reportFileName: 'report.html',
unstableThreshold: 0]);
}
}
}
stage('Jmeter') {
agent { node { label 'jmeter' } }
steps {
sh "rm -rf /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/*"
checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'binbin', url: 'https://github.com/zbbkeepgoing/springboot-demo.git']]])
sh "mkdir -p /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/output"
sh "jmeter.sh -n -t /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/demo.jmx -l /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/demo.jtl -j /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/demo.log -e -o /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/output"
step([$class: 'ArtifactArchiver', artifacts: 'jmeter/*,jmeter/output/*'])
perfReport "jmeter/demo.jtl"
}
}
stage('Ansible') {
agent { node { label 'ansible' } } //指定ansible節點進行執行
steps {
dir(env.WORKSPACE){ //指定在該Pipeline的工作目錄下面執行
checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'binbin', url: 'https://github.com/zbbkeepgoing/springboot-demo.git']]]) //拉取最新的程式碼
sh 'cd ansible ; ansible-playbook -i hosts deploy.yml ' //ansible-playbook執行ansible指令碼
}
}
}
}
}
-
新建Pipeline
-
執行Pipeline
Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] stage
[Pipeline] { (Preparation)
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible
[Pipeline] {
[Pipeline] checkout
Cloning the remote Git repository
Cloning repository https://github.com/zbbkeepgoing/springboot-demo.git
......
> git checkout -f 1aa03a20b88565b775a23a759fde2701cabe8592
Commit message: "Update deploy.sh"
First time build. Skipping changelog.
......
[Pipeline] { (Build)
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible
[Pipeline] {
[Pipeline] dir
Running in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible
[Pipeline] {
[Pipeline] sh
[CI+Sonar+Sh+Robot+Jmeter+Ansible] Running shell script
+ mvn clean install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building demo 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
......
Results :
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO]
......
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 39.116 s
[INFO] Finished at: 2018-10-28T03:16:46+00:00
[INFO] Final Memory: 29M/69M
[INFO] ------------------------------------------------------------------------
[Pipeline] junit
Recording test results
[Pipeline] sh
[CI+Sonar+Sh+Robot+Jmeter+Ansible] Running shell script
+ mv target/sample-0.0.1-SNAPSHOT.jar target/sample.jar
......
[Pipeline] { (Sonarqube)
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible
[Pipeline] {
[Pipeline] dir
Running in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible
[Pipeline] {
[Pipeline] sh
[CI+Sonar+Sh+Robot+Jmeter+Ansible] Running shell script
+ mvn sonar:sonar -Dsonar.host.url=http://192.168.88.130:9000 -Dsonar.login=65607ba9d0f54590cf55fe8e60134fb5e87c557d
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building demo 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- sonar-maven-plugin:3.5.0.1254:sonar (default-cli) @ sample ---
......
[INFO] Analysis report generated in 407ms, dir size=51 KB
[INFO] Analysis reports compressed in 41ms, zip size=16 KB
[INFO] Analysis report uploaded in 306ms
[INFO] ANALYSIS SUCCESSFUL, you can browse http://192.168.88.130:9000/dashboard/index/com.dxc.ddccloud:sample
[INFO] Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
[INFO] More about the report processing at http://192.168.88.130:9000/api/ce/task?id=AWa4rTobNiNVbk1zeDos
[INFO] Task total time: 22.148 s
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 34.289 s
[INFO] Finished at: 2018-10-28T03:17:32+00:00
[INFO] Final Memory: 33M/152M
[INFO] ------------------------------------------------------------------------
.......
[Pipeline] { (Deploy)
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible
[Pipeline] {
[Pipeline] dir
Running in /var/jenkins_home/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible
[Pipeline] {
[Pipeline] sh
[CI+Sonar+Sh+Robot+Jmeter+Ansible] Running shell script
+ sshpass -p [email protected] scp -o StrictHostKeychecking=no target/sample.jar [email protected]:/opt/ansible
[Pipeline] sh
[CI+Sonar+Sh+Robot+Jmeter+Ansible] Running shell script
+ sshpass -p [email protected] scp -o StrictHostKeychecking=no deploy.sh [email protected]:/opt/ansible
[Pipeline] sh
[CI+Sonar+Sh+Robot+Jmeter+Ansible] Running shell script
+ sshpass -p [email protected] ssh -o StrictHostKeychecking=no [email protected] bash /opt/ansible/deploy.sh
process not started
[Pipeline] sh
[CI+Sonar+Sh+Robot+Jmeter+Ansible] Running shell script
+ sleep 8s
......
[Pipeline] { (Robot Framework)
[Pipeline] node
Still waiting to schedule task
Jenkins doesn’t have label robot
Running on robot-00000sz2sjfm7 on 192.168.88.130robot in /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible
[Pipeline] {
[Pipeline] dir
Running in /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible
[Pipeline] {
[Pipeline] checkout
Cloning the remote Git repository
Cloning repository https://github.com/zbbkeepgoing/springboot-demo.git
.......
Commit message: "Update deploy.sh"
[Pipeline] sh
[CI+Sonar+Sh+Robot+Jmeter+Ansible] Running shell script
+ pybot -d /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible robot/demo.robot
==============================================================================
Demo
==============================================================================
Demo | PASS |
------------------------------------------------------------------------------
Baidu | PASS |
------------------------------------------------------------------------------
Demo | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
Output: /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/output.xml
Log: /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/log.html
Report: /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/report.html
[Pipeline] step
Robot results publisher started...
-Parsing output xml:
Done!
-Copying log files to build dir:
Done!
-Assigning results to build:
Done!
-Checking thresholds:
Done!
Done publishing Robot results.
......
[Pipeline] { (Jmeter)
[Pipeline] node
Still waiting to schedule task
Jenkins doesn’t have label jmeter
Running on jmeter-00000vxdsfpfp on 192.168.88.130jmeter in /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible
[Pipeline] {
[Pipeline] sh
[CI+Sonar+Sh+Robot+Jmeter+Ansible] Running shell script
+ rm -rf /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/*
[Pipeline] checkout
Cloning the remote Git repository
Cloning repository https://github.com/zbbkeepgoing/springboot-demo.git
.......
Commit message: "Update deploy.sh"
[Pipeline] sh
[CI+Sonar+Sh+Robot+Jmeter+Ansible] Running shell script
+ mkdir -p /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/output
[Pipeline] sh
[CI+Sonar+Sh+Robot+Jmeter+Ansible] Running shell script
+ jmeter.sh -n -t /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/demo.jmx -l /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/demo.jtl -j /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/demo.log -e -o /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/output
Oct 28, 2018 3:23:03 AM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Creating summariser <summary>
Created the tree successfully using /opt/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible/jmeter/demo.jmx
Starting the test @ Sun Oct 28 03:23:05 UTC 2018 (1540696985655)
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary + 29 in 00:00:26 = 1.1/s Avg: 1464 Min: 18 Max: 9052 Err: 0 (0.00%) Active: 2 Started: 2 Finished: 0
summary + 18 in 00:00:30 = 0.6/s Avg: 3318 Min: 19 Max: 10053 Err: 0 (0.00%) Active: 2 Started: 2 Finished: 0
summary = 47 in 00:00:57 = 0.8/s Avg: 2174 Min: 18 Max: 10053 Err: 0 (0.00%)
summary + 13 in 00:00:20 = 0.6/s Avg: 3878 Min: 10 Max: 10063 Err: 0 (0.00%) Active: 0 Started: 2 Finished: 2
summary = 60 in 00:01:17 = 0.8/s Avg: 2543 Min: 10 Max: 10063 Err: 0 (0.00%)
Tidying up ... @ Sun Oct 28 03:24:24 UTC 2018 (1540697064881)
... end of run
[Pipeline] archiveArtifacts
Archiving artifacts
[Pipeline] perfReport
Performance: Recording JMeterCsv reports 'jmeter/demo.jtl'
Performance: Parsing JMeter report file '/var/jenkins_home/jobs/CI+Sonar+Sh+Robot+Jmeter+Ansible/builds/1/performance-reports/JMeterCSV/demo.jtl'.
Performance: No threshold configured for making the test unstable
Performance: No threshold configured for making the test failure
Performance: File demo.jtl reported 0.0% of errors [SUCCESS]. Build status is: SUCCESS
......
[Pipeline] { (Ansible)
[Pipeline] node
Still waiting to schedule task
ansible-00000y1xxwwx7 is offline
Running on ansible-00000y1xxwwx7 on 192.168.88.130ansible in /opt/tmp/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible
[Pipeline] {
[Pipeline] dir
Running in /opt/tmp/workspace/CI+Sonar+Sh+Robot+Jmeter+Ansible
[Pipeline] {
[Pipeline] checkout
Cloning the remote Git repository
Cloning repository https://github.com/zbbkeepgoing/springboot-demo.git
......
Commit message: "Update deploy.sh"
[Pipeline] sh
[CI+Sonar+Sh+Robot+Jmeter+Ansible] Running shell script
+ cd ansible
+ ansible-playbook -i hosts deploy.yml
PLAY [deploy demo jar] *********************************************************
TASK [demo : Copy sample.jar to remote vm] *************************************
changed: [192.168.88.128]
TASK [demo : Get running sample.jar] *******************************************
changed: [192.168.88.128]
TASK [demo : Kill running sample.jar] ******************************************
TASK [demo : wait_for] *********************************************************
TASK [demo : Force kill stuck sample.jar] **************************************
TASK [demo : Sleep 10s] ********************************************************
changed: [192.168.88.128]
TASK [demo : Deploy sample.jar] ************************************************
changed: [192.168.88.128]
PLAY RECAP *********************************************************************
192.168.88.128 : ok=4 changed=4 unreachable=0 failed=0
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] End of Pipeline
Finished: SUCCESS
- 檢視ansible部署節點的應用
以上就是我們Ansible自動化部署整合Pipeline的所有內容。後續我們將進行一個小總結，總結之前所有工具和技術整合的流程。