Jenkins 如何创建动态的并发stage?

936次阅读
没有评论
Jenkins

以前公司曾经有段时间,一些开发在做微前端的技术调研时,曾让我帮忙写个pipeline,一个前端工程下有多个子组件,而博主构建时需要去到每个目录中进行npm操作,随着子组件的增加,博主就需要重写pipeline增加新组件构建,属实麻烦

起初是一个一个dir进去执行,类似如下:

目前初步想法就是使用parallel并行任务来处理多个组件构建

pipeline {
    agent any
    stages {
        stage('parallel') {
            steps {
                script {
                    def tasks = [:]
                    def componentlist =  ["alpha", "beta", "gamma", "delta", "epsilon", "zeta", "eta", "theta", "iota", "kappa"]
                    for (component in componentlist) {
                        def component_inside = "${component}"
                        tasks["${component_inside}"] = {
                                stage("${component_inside}") {
                                    echo "${component_inside}"
                                }
                        }
                    }
                    parallel tasks
                }
            }
        }
    }
}
Jenkins

目前效果如上,可以通过一个指定列表来动态扩展stage,但依然还要人工来写这个componentlist,依然很麻烦******,一顿搜索后可以将这个列表从文件中读取(readFile),而这个文件则由开发人员提交上代码仓库,这样博主就不需要修改pipeline了,完全由开发自己决定。。。

pipeline {
    agent any
    stages {
        stage() {
            git: xxxx@xxxxx branch: xxxx
        }
        stage('parallel') {
            steps {
                script {
                    def tasks = [:]
                    //def componentlist =  ["alpha", "beta", "gamma", "delta", "epsilon", "zeta", "eta", "theta", "iota", "kappa"]
                    def componentlist = readFile('componentlist').replace("\n", "").split(',')
                    for (component in componentlist) {
                        def component_inside = "${component}"
                        tasks["${component_inside}"] = {
                                stage("${component_inside}") {
                                    echo "${component_inside}"
                                }
                        }
                    }
                    parallel tasks
                }
            }
        }
    }
}

而该componentlist文件内容如下

cat componentlist
alpha,beta,gamma,delta

一段时间后,问题又来了,因为并发构建,随着组件数量增加,博主的jenkins服务器吃不消,本来jenkins服务设置的按核心数量来运行任务数,现在这个任务直接并发多个运行,需要给他限制下并发2个组件构建,不然扛不住

pipeline {
    agent any
    stages {
        stage('parallel') {
            steps {
                script {
                    def componentlist =  ["alpha", "beta", "gamma", "delta", "epsilon", "zeta", "eta", "theta", "iota", "kappa"]
                    def parallelBatches = componentlist.collate(2);
                    
                    for (parallelBatch in parallelBatches) {
                        def tasks = [:]
                        for (component in parallelBatch) {
                            def component_inside = "${component}"
                            tasks["${component_inside}"] = {
                                stage("${component_inside}") {
                                    echo "${component_inside}"
                                    sh "sleep 5"
                                }
                            }
                        }
                        parallel tasks
                    }
                }
            }
        }
    }
}
xadocker
版权声明:本站原创文章,由 xadocker 2020-09-11发表,共计1668字。
转载说明:除特殊说明外本站文章皆由CC-4.0协议发布,转载请注明出处。
评论(没有评论)
验证码
载入中...
0.264