Jenkins / Hudson getting the overall status in the upstream job from the downstream jobs

Few days ago i asked in http://www.stackoverflow.com this question:

http://stackoverflow.com/questions/6141003/jenkins-hudson-upstream-job-does-not-get-the-status-ball-color-of-the-downstrea

I wanted to get the result of the upstream job depending on the result of the downstream jobs. I mean:

If in the upstream job i get a “stable” result but if in one of the downstream jobs i get an “unstable” or “failed” result i want to get the worst status in my upstream job.

In my case, i wanted to speed up the test execution paralellizing the build in different downstream jobs, therefore, i wanted to have the overall result of the build in the main job (the upstream) in order to see the result of the complete build in a single point.

I couldn’t find any good solution, only workarounds. So i started to investigate and discovered the best Jenkins / Hudson plugin i have ever seen: the Groovy Postbuild plugin

https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Postbuild+Plugin

With this plugin you can do almost whatever you want with Jenkins / Hudson. It allows you to execute a Groovy script in the post build phase and offers you the “build” and “hudson” Java objects.

The solution for my problem was a simple Groovy script added in the downstream jobs. This code gets the upstream job, access to the last build, check the result and if its result is better than the downstream one, it updates it.

This is the piece of code:

upstreamBuilds = manager.build.getUpstreamBuilds();
upstreamJob = upstreamBuilds.keySet().iterator().next();
lastUpstreamBuild = upstreamJob.getLastBuild();
if(lastUpstreamBuild.getResult().isBetterThan(manager.build.result)) {
    lastUpstreamBuild.setResult(manager.build.result);
}

You can easily adapt this code with the requirements of your configuration.

Advertisements

19 thoughts on “Jenkins / Hudson getting the overall status in the upstream job from the downstream jobs

  1. Hello,

    I’ve been working on this topic but once I tried your script it didn’t run, I’ve tried different combinations but I couldn’t find the key to execute it properly.

    Jenkins version: 1.414
    Groovy Postbuild Plugin 1.4

    Source
    upstreamBuilds = manager.build.getUpstreamBuilds();
    upstreamJob = upstreamBuilds.keySet().iterator().next();
    lastUpstreamBuild = upstreamJob.getLastBuild();
    if(lastUpstreamBuild.getResult().isBetterThan(manager.build.result)) {
    lastUpstreamBuild.setResult(manager.build.result);
    }

    error:
    java.util.NoSuchElementException
    at java.util.HashMap$HashIterator.nextEntry(HashMap.java:796)
    at java.util.HashMap$KeyIterator.next(HashMap.java:828)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229)
    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:52)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:43)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
    at Script1.run(Script1.groovy:2)
    at groovy.lang.GroovyShell.evaluate(GroovyShell.java:561)
    at groovy.lang.GroovyShell.evaluate(GroovyShell.java:536)
    at org.jvnet.hudson.plugins.groovypostbuild.GroovyPostbuildRecorder.perform(GroovyPostbuildRecorder.java:262)
    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
    at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:663)
    at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:638)
    at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:616)
    at hudson.model.Build$RunnerImpl.post2(Build.java:161)
    at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:585)
    at hudson.model.Run.run(Run.java:1399)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:145)

    I;ve also tried using a similar groovy script but it seems I cannot upgrade the parent job>

    manager.listener.logger.println(manager.build.parent.getLastBuild())
    manager.buildUnstable()
    if(manager.build.result.isWorseOrEqualTo(hudson.model.Result.UNSTABLE))
    {
    manager.build.parent.lastBuild.buildUnstable()
    }

    error:

    groovy.lang.MissingMethodException: No signature of method: hudson.model.FreeStyleBuild.buildUnstable() is applicable for argument types: () values: []
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:54)
    at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:48)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:43)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
    at Script1.run(Script1.groovy:7)
    at groovy.lang.GroovyShell.evaluate(GroovyShell.java:561)
    at groovy.lang.GroovyShell.evaluate(GroovyShell.java:536)
    at org.jvnet.hudson.plugins.groovypostbuild.GroovyPostbuildRecorder.perform(GroovyPostbuildRecorder.java:262)
    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
    at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:663)
    at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:638)
    at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:616)
    at hudson.model.Build$RunnerImpl.post2(Build.java:161)
    at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:585)
    at hudson.model.Run.run(Run.java:1399)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
    at hudson.model.ResourceController.execute(Resourc

    Thanks

  2. Hello,

    The first error executing my Groovy code is because the iterator is empty because it cannot find the upstream build.

    The Groovy code is executed as a post build action, like the recording of the fingerprints used to join the upstream and the downstreams jobs. In your case, the Groovy code is executed before recording the fingerprints, therefore, at the moment of the execution of the Groovy script, there is no upstream project yet, so the iterator is empty and you get that error.

    There is a ticket opened in the Jenkins and Hudson issue tracker to add a way to specify the order of the postbuild actions to avoid cases like this one.

    https://issues.jenkins-ci.org/browse/JENKINS-7408

    As a workaround that might work (not sure), change the order manually.
    The job configurations are stored in the master node in XML files inside the directory “jobs”, get the XML file of that job and change the order of the postbuild actions at the bottom of the file, setting the Groovy script at the end.

    Then restart Jenkins and probably the next time the postbuild order executions will be different.

    Maybe you should add a check in the Groovy code to not get that error, check if the iterator is empty. Something like:

    upstreamBuilds = manager.build.getUpstreamBuilds();
    if(upstreamBuilds.keySet().iterator().hasNext()) {
    upstreamJob = upstreamBuilds.keySet().iterator().next();
    lastUpstreamBuild = upstreamJob.getLastBuild();
    if(lastUpstreamBuild.getResult().isBetterThan(manager.build.result)) {
    lastUpstreamBuild.setResult(manager.build.result);
    }
    }

    • Hi Victor I’m seeing the first error (NoSuchElementException). In my configuration, I only have Groovy Postbuild in post build action section. Is there anything else that needs to be done (like the Fingerprint you said)? Thanks.

      • Hello Peter,

        Yes, the fingerprint is necessary because it’s the way to connect the downstream jobs with the upstream jobs, otherwise, you will get the NoSuchElementException always since there won’t be any upstream or dowstream jobs to look for.
        In my previous post i explained how to set it up

        Regards!

  3. Is it the case that the upstream job will be flagged as stable (or passed) and at some point later, after all downstream jobs have finished, it will become failed?

    This bahaviour will make an automated analysis very hard. Also a job that can change it’s status after it has completed is not the result of a good design, right?

    Is there a way to have the upstream job wait for all downstream jobs before it completes?

    Thanks for your help,
    Norman

    • Yes, it will be marked as failed if any of the downstream jobs fail but in my case, i don’t consider the job as completed until its downstream jobs are completed.

      Think something like the upstream job is a code builder and the downstream jobs are the test executors for the built code and you have more than one to speed the execution up making it parallel in different machines. If you want to see if all the tests passed succesfully you can see in a confortable way the state of the tests in a single point ( the upstream job state ) instead of review all the states of the downstream jobs.

      Therefore you will need to block the upstream job till all the downstream finish, there is an option in the Hudson/Jenkins project configuration in the section “Advanced Project Options” you need to activate: “Block build when downstream project is building”

      Anyway, as you say, the analysis could be hard, maybe this plugin can help you

      https://wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin

      It shows a chart with the workflow of the upstream/downstream jobs

      Victor

    • Well, i can’t help you if i don’t know how you have your Jenkins configuration, add traces to the Groovy code to check what’s going on, spend some time debugging.
      Be sure you can access to the upstream build and the downstream build and they are connected by the fingerprints

      • Hi
        I successfully use groovy script:

        upstreamBuilds = manager.build.getUpstreamBuilds();
        upstreamJob = upstreamBuilds.keySet().iterator().next();
        lastUpstreamBuild = upstreamJob.getLastBuild();
        if(lastUpstreamBuild.getResult().isBetterThan(manager.build.result)) {
        lastUpstreamBuild.setResult(manager.build.result);
        }

        ,but it update one upstream project
        I need update more upstreams,but dont know groovy and have no Api for jenkins groovy plugin
        Please help me upgrade code for update all upstream projects

        Thanks

  4. I think you’re misinterpreting the meaning of “Block build when downstream project is building”. The help test says, ” When this option is checked, Jenkins will prevent the project from building when a child of this project is in the queue, or building. The children include the direct as well as the transitive children.” That sounds more like the parent build will simply wait to run if one of its child projects are building. Jenkins regards the parent build as complete before running the child projects (Build Other Projects is a “Post-build Action”). The setting you’re talking about won’t extend the parent build to last longer.

    • Yes, you are right. This option doesn’t make the upstream build to be uncompleted until the children finish, as you say, Jenkins regards it as completed even though blocks it to not start new builds until all the children finish.
      But when i wrote that comment, i think i wanted to mean that the Groovy code will work only if you have this option checked, because that Groovy code gets the upstream build with

      upstreamJob.getLastBuild()

      and if that block option is not checked, the last build might not be the build that triggered the children builds, so the Groovy code will update the build state wrongly.

      So, i don’t know how to set an uncomplete state in the upstream build until the children finish and probably this is not possible because as far as i know, if you set an uncomplete state in a build it is considered as still building, and actually, it isn’t.

      If any of you have the solution for this, please tell me!

      Thanks for your feedback

  5. Your example updates the status of the latest build only.
    I would like to consider concurrent builds, in which I would like to update a particular build number status, no the latest one, and the possibility of re-running a downstream run.

    If a downstream failed, and one re-run it (with the same parameters), it seems impossible to change the status from FAILED to anything else (i.e. STABLE or FAILED)

    Consider the scenario when something is offline (e.g. your internal maven repo), downstream can fail, however this is not a real failure since re-running the exact same build can succeed but won’t change the overall status.

    I can get a given build by using getBuildByNumber, however I can’t change the status if it’s set to FAILED.

    Any idea ?

    • Hi, sorry for the late response.

      At this moment i’m not working with this stuff anymore, so, to be able to give you a clear solution i would need to set the full environment up…

      But, if i remember correctly, i reckon that you can change the status if a downstream build is re-run.
      If the first build is marked as FAILED because a downstream job failed, but you re-run it and finished successfully in the post build code you can go to the upstream build and change it status, probably, it didn’t work for you because given that the build status is already saved you will need to re-save it. When i investigated the Java Jenkins API i remember something like a config.save option. Maybe that could fix your problem.

      I’m sorry for not being more helpful at this moment

      Regards

  6. I want to display the statuses of down stream project when I select a project in Hudson. From config.xml I am able to get the names of downstream projects but not the statuses. Where the statuses will be stored for down stream projects of a project.

    Thanks
    Surya

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s