jenkins:實現Jenkinsfile與Json的轉換
阿新 • • 發佈:2021-02-09
### 實現Jenkinsfile與Json的轉換
[TOC]
最近在做個需求,需要支援Jenkinsfile和json的轉換。
### 方法1:使用現有的jenkins外掛
參考的是[這篇文章](https://www.cnblogs.com/w1570631036/p/11726599.html)。下面介紹一下將外掛打包成映象的步驟:
- 本地安裝jdk和maven,jdk建議採用的版本為8(該工程會依賴一個名為`tools.jar`的包,jdk 9之後移除了該包)
如果本地沒有找到`tools.jar`,可以下載一個1.8版本的jdk,然後在`pom.xml`中增加如下依賴
```xml
```
- clone [pipeline-model-definition-plugin](https://github.com/jenkinsci/pipeline-model-definition-plugin)工程
- 在/root/.m2/目錄下建立`settings.xml`,內容來自[Jenkins官方](https://wiki.jenkins.io/display/JENKINS/Plugin+tutorial):
```xml
```
- 執行 `mvn install`進行編譯
- 由於主要用到的是Jenkinsfile和json之間的轉換關係,因此主要用的是如下兩個REST API:
- **Conversion to JSON representation from Jenkinsfile**
- *URL*: `JENKINS_URL/pipeline-model-converter/toJson`
- *Parameters*: `jenkinsfile` - the `Jenkinsfile` contents
- *Info*: Takes a `Jenkinsfile` and converts it to the JSON representation for its `pipeline` step.
- *Returns*: JSON with a `result` field that will either be `success` or `failure`. If `success`, the JSON representation will be in the `json` field. If `failure`, there'll be an additional array in the `errors` field of the error messages encountered.
- **Conversion to Jenkinsfile from JSON representation**
- *URL*: `JENKINS_URL/pipeline-model-converter/toJenkinsfile`
- *Parameters*: `json` - the JSON representation of the model
- *Info*: Takes the JSON representation of the model and converts it to the contents for a `Jenkinsfile` invoking the `pipeline` step.
- *Returns*: JSON with a `result` field that will either be `success` or `failure`. If `success`, the `Jenkinsfile` contents will be in the `jenkinsfile` field. If `failure`, there'll be an additional array in the `errors` field of the error messages encountered.
上述兩個API在`pipeline-model-definition-plugin/pipeline-model-definition`目錄下,因此在該目錄下直接執行:`mvn hpi:run -Dhost=0.0.0.0 -Djetty.port=8080`即可。
- 將json轉換為Jenkinsfile的操作如下:
完整的返回值如下:
```json
{
"status": "ok",
"data": {
"result": "success",
"json": {
"pipeline": {
"stages": [
{
"name": "Hello",
"branches": [
{
"name": "default",
"steps": [
{
"name": "echo",
"arguments": [
{
"key": "message",
"value": {
"isLiteral": true,
"value": "Hello World"
}
}
]
}
]
}
]
}
],
"agent": {
"type": "any"
}
}
}
}
}
```
- 將Jenkinsfile轉換為json的操作如下
- 製作容器映象時,只需要將本地工程和`/root/.m2`上傳到容器,生成對應的映象即可,下面Dockerfile假設生成的映象為`pipeline-model-definition-plugin:latest`
```dockerfile
FROM pipeline-model-definition-plugin:latest
WORKDIR /usr/pipeline-model-definition-plugin/pipeline-model-definition
ENV PATH=$PATH:/usr/local/bin/maven-3.6.3/bin
ENTRYPOINT ["sh", "-c", "mvn hpi:run -Dhost=0.0.0.0"]
```
我自己打包了一個映象:`docker pull quay.io/woodliu/pipeline-model-definition-plugin`
需要注意的是,本外掛提供的轉換API toJenkinsfile和toJson並不是萬能的,只能支援jenkins標準的引數[型別](https://www.jenkins.io/doc/book/pipeline/syntax/#parameters),例如對於`gitParameter`這樣的引數就無法解析(擴充套件功能),一種解決方式是獨立解析擴充套件的引數,然後將其插入解析好的標準JenkinsFile中;另外一個方式就是寫一個jenkinsfile的解析器。
#### 參考
- mvn hpi的命令可以參考[官方文件](https://jenkinsci.github.io/maven-hpi-plugin/plugin-info.html)
- 可以執行`mvn hpi:hpi`生成對應的`hpi`檔案,如:
`/pipeline-model-definition-plugin/pipeline-model-definition/target/pipeline-model-definition.hpi`
### 方法2:解析原生的jenkinsfile檔案
在GitHub上有一個支援jenkinsfile解析的[專案](https://github.com/rtyler/jdp),該專案使用rust的[pest](https://github.com/pest-parser/pest) crate來編寫jenkinsfile的語法,支援對jenkinsfile的格式驗證。Pest官方文件中給出了一個非常好的對json語法的解析[例子](https://pest.rs/book/examples/json.html),主要是使用遞迴的方式來解析語法。
pest官方提供了一個[編輯器](https://pest.rs/#editor),可以使用該編輯器檢視經過pest解析之後的欄位,對了解pest的工作方式非常有用。如,使用jdp專案提供的[pest檔案](https://github.com/rtyler/jdp/blob/main/src/pipeline.pest)解析如下jenkinsfile:
```groovy
pipeline {
agent {
docker {
reuseNode true
image 'maven:3-alpine'
label 'my-defined-label'
args '-v /tmp:/tmp'
registryUrl 'https://myregistry.com/'
registryCredentialsId 'myPredefinedCredentialsInJenkins'
}
}
stages {
stage('Build') {
steps { sh 'make' }
}
}
}
```
對應的解析結果如下:
```
- preceeding_junk: ""
- opening_brace: "{"
- agentDecl > agentBlock
- opening_brace: "{"
- dockerAgent
- opening_brace: "{"
- bool: "true"
- string > single_quoted
- single_quote: "\'"
- inner_single_str: "maven:3-alpine"
- single_quote: "\'"
- string > single_quoted
- single_quote: "\'"
- inner_single_str: "my-defined-label"
- single_quote: "\'"
- string > single_quoted
- single_quote: "\'"
- inner_single_str: "-v /tmp:/tmp"
- single_quote: "\'"
- string > single_quoted
- single_quote: "\'"
- inner_single_str: "https://myregistry.com/"
- single_quote: "\'"
- string > single_quoted
- single_quote: "\'"
- inner_single_str: "myPredefinedCredentialsInJenkins"
- single_quote: "\'"
- closing_brace: "}"
- closing_brace: "}"
- stagesDecl
- opening_brace: "{"
- stage
- string > single_quoted
- single_quote: "\'"
- inner_single_str: "Build"
- single_quote: "\'"
- opening_brace: "{"
- stepsDecl
- opening_brace: "{"
- step > simple_step
- IDENT: "sh"
- args > string > single_quoted
- single_quote: "\'"
- inner_single_str: "make"
- single_quote: "\'"
- closing_brace: "}"
- closing_brace: "}"
- closing_brace: "}"
- closing_brace: "}"
- ending_junk: ""
- EOI: ""
```
> Pest語法重點標註:
>
> - 當使用**靜默**規則時,解析結果中將不會出現該規則欄位。當解析下面規則時,解析結果中將不會存在silent,即`parsed.as_rule()` 中不會存在silent
>
> ```
> silent = _{ ... }
> ```
>
> - 當使用**原子**語法時,整個規則體將視為一個規則,如double_quoted = ${ (quote ~ inner_double_str ~ quote) },在解析時會將`quote ~ inner_double_str ~ quote`視為一個規則,而不是三個。這有利於獲取一段完整的字串。
>
> ```
> atomic = @{ ... }
> compound_atomic = ${ ... }
> ```
我嘗試使用該專案解析jenkinsfile,但發現實現起來太過複雜,且jenkinsFile的語法也是一言難盡。如下,當step中帶括號和不帶括號混用時會導致解析錯誤。
```groovy
steps {
echo 'test'
dir('command') {
sh "sh ./saas.sh ${params.channel} ${params.buildType} "
}
}
```
有精力的大神可以在此基礎上實現解析JenkinsFile的功能。
#### 參考
- pest[文件](https://pest.rs/book/int