Some days ago i wrote a post about how to get the overall status in the upstream job from the downstream jobs in Hudson or Jenkins (https://fatalfailure.wordpress.com/2011/06/14/jenkins-hudson-getting-the-overall-status-in-the-upstream-job-from-the-downstream-jobs/).
A user (v22 ) test my code and had some problems, and actually, there is a problem.
As i said in the comment reply, there is a problem in the execution order of the post build actions, this is not deterministic and depends on how the job configuration is stored in a XML file located in the master node.
The relationship between the upstream jobs and the downstream jobs is ready when the fingerprints are recorded. This action is done on a post build action, like the Groovy execution. If the Groovy execution is performed before the fingerprint recording, there is not upstrem job set yet, therefore the Groovy code fails.
I proposed in that reply a workaround to change the order execution of the post build actions, but this is not working as i expected, so there is not a way to force the Groovy execution after the fingerprint recording.
There is an open ticket in the Jenkins and Hudson issue tracker to add a way to specify the post build actions executions: https://issues.jenkins-ci.org/browse/JENKINS-7408
But everything has a solution, and a solution that always works 😉
There is a plugin called “Join Plugin” (https://wiki.jenkins-ci.org/display/JENKINS/Join+Plugin) that let you execute jobs after all the downstream jobs finish.
The only thing you need to do is configure the upstream job with the Join plugin setting a new job (“Join job”) to be executed after all the downstreams are finished. In that “Join job” you need to add the Groovy code with the Groovy postbuild plugin.
The code here would be more sophisticated, but not very complex.
Take a look to the code i wrote in my post and edit it, i’m not going to write the new code here, i’ll let you investigate a bit. But here are some clues:
- Instead of use getUpstreamBuilds(), you will need getTransitiveUpstreamBuilds()
- Once you have the upstream job, use getDownstreamProjects() to get the downstream jobs
- Loop over the downstream jobs checking their states and update the upstream job accordingly
You can remove the Groovy code in the downstream jobs.
Fatal failure 
Could you kindly list the code for this one? I don’t see a ‘getDownstreamProjects()’ method in the AbstractBuild javadoc: http://javadoc.jenkins-ci.org/index.html?hudson/model/AbstractBuild.html
Also, with the join job technique do we need to use fingerprinting?
Yes, you don’t have that method in the AbstractBuild but in AbstractProject, and you can get it iterating over the keys of the Map returned by getTransitiveUpstreamBuilds().
The pseudocode would be something like:
upstreamBuilds = manager.build.getTransitiveUpstreamBuilds()
while(upstreamBuilds.keySet.iterator.hasNext()) {
upstreamJob = upstreamBuilds.keySet.iterator.next()
downstreamJobs = upstreamJob.getDownstreamProjects()
…
}
About the fingerprinting, the join plugin sets a job to be run after all the downstream jobs finish, therefore, you will need fingerprinting to set which are the downstream jobs. But, you don’t need fingerprinting to “connect” the upstream job with the join job.
Let’s see an example:
JobA is the main job, the upstream
JobB and JobC are the downstream jobs
JobD is the join job.
JobA run and after it finish, the downstream jobs (JobB and JobC) start running because are triggered as postbuild actions in the configuration of JobA.
JobB and JobC must be “connected” with JobA via fingerprinting, otherwise, the jobs will be run separately and they won’t be downstream jobs, they will only be “jobs”.
After JobB and JobC finish, the join job (JobD) starts and executes that Groovy code. This JobD is run because it’s configured in JobA with the Join Plugin. You don’t need fingerprinting to connect JobA with JobD.
If you have any other doubt, don’t hesitate to ask me
Regards!
Victor
Hi Victor,
Thanks so much for the reply.
I created four jobs Test_Fingerprint_1, Test_Fingerprint_2, Test_Fingerprint_3 and Test_Fingerprint_Join that bear the following relation:
TF1 –> TF2 –> TF3
|
——————-> TFJ
I’m new to fingerprinting so I’m not sure if I’m doing the fingerprinting operation correctly. I’ll list what I’ve setup:
TF1 creates a file called f1 which it archives and fingerprints
TF2 copies the artifact f1 and also creates a file f2 from TF1. It archives and fingerprint both f1 and f2
TF3 copies the artifacts f1 and f2 from TF2 and records fingerprint on both.
Doing this, I was able to obtain the correct relationship in the Jenkins ‘Project Relationship’ page.
After this I wrote the following groovy script on the TFJ job:
upstreamBuilds = manager.build.getTransitiveUpstreamBuilds();
upstreamJob = upstreamBuilds.iterator().hasNext();
manager.createSummary(“warning.gif”).appendText(“$upstreamJob”, false, false, false, “red”)
And I get the result ‘false’ in the build page every time? What could I be doing wrong? I tried listing upstreamBuilds.size() and it turns out to be zero.
http://imageshack.us/photo/my-images/196/falsej.png/
Please advise.
You are getting “false” because hasNext() returns boolean, anyway, it should returns “true”. Maybe the fingerprinting is not done correctly.
In the jobs connected by a fingerprint, you need to check if they are connected. For example, go to one of the builds of the TF1 job and click in “see fingerprints” in the right menu. You will see the fingerprints you created and in the “Original Owner” column must be “this owner” ( that means that the archive to fingerprint needs to be created during the build execution, Jenkins checks the creation date to be sure the archive to fingerprint is valid ), then you can click in more details near the fingerprint archive name and you will see what jobs are using it, if there is no jobs, something is wrong. For this example, you will see the usage of TF1 and TF2.
The fingerprint technique is not very well documented so maybe, if i have free time, i’ll write a new entry with a brief tutorial in this blog of how to set the fingerprinting in Jenkins/Hudson, step by step.
It could be useful for many people
I did went to the ‘See Fingerprints’ link in a build of TF1 and saw this:
http://imageshack.us/photo/my-images/24/recordedfingerprints.png/
And in ‘more details’:
http://imageshack.us/photo/my-images/651/seefingerprint.png/
This is as it should b I believe since the artifact f1 is being copied by the downstream jobs.
Yes, the fingerprinting operation has very poor documentation. It would be nice to see a good tutorial.
I did go*
I tried again, this time with two jobs TF1 and Tf2 and the join Job TFJ only and I’m still getting the error.
Be sure the Groovy code is in the join job.
I don’t know what is done wrong if i don’t see the jobs configuration. I think the best thing you can do is debugging the Groovy code, also known as, adding traces. You can use “manager.listener.logger.println(…)” to write traces in the build log and see what the objects contain, try different methods listed in the Javadoc instead of getTransitiveUpstreamBuilds(), i’m sure you will get any clue of what’s going on.
You also can add Groovy code in the downstream job to see if it’s able to access to the upstream job to check if everything is well connected.
I cannot tell you anything else with that information, the fingerprinting seems to be ok, but there should be something weird anywhere.
Thanks again for your help. I’ll try the tips you mentioned.
Hi Victor,
Inspired by your code, I tried another approach to this issue that does not rely on fingerprints.
I used the getProject() and getUpstreamProjects() function to get a complete list of all projects in the workflow. In the arrangement I have setup all of the downstream jobs will be triggered after build completes on its respective parent job. So I just got the last build of every upstream project and performed the check for instability or failure.
Here’s the code:
def buildList = [];
proj = manager.build.getProject();
// While there are upstream projects
while (proj.iterator().hasNext()){
temp = proj.lastBuild; // Get last build of upstream project
buildList.add(temp); // and add it to the build list
proj = proj.getUpstreamProjects()[0]; // getUpstreamProjects() returns a list with one element
}
listSize = buildList.size();
if (listSize > 1){
// Invert the list since list will be in “last jobs first” order
for (i = 0; i < listSize/2; i++){
temp = buildList[i];
buildList[i] = buildList[listSize – 1 – i];
buildList[listSize – 1 – i] = temp;
}
manager.listener.logger.println("List of builds in present workflow: $buildList");
for (i = 1; i < listSize; i++){
if (buildList[0].getResult().isBetterThan(buildList[i].getResult())){
buildList[0].setResult(buildList[i].getResult());
break;
}
}
}
Now, there is a slight problem. The setResult function will set the status of the upstream jobs as it should but once I restart the Jenkins service the status is reverted back to the original status i.e the modification made by setResult() does not persist across Jenkins service restarts.
I figured it out myself. All I had to do was add the line:
buildList[0].save();