it is not often, but it appears sometimes the pipeline gets stuck when trying to execute command in a container.
ex: https://jenkins-continuous-infra.apps.ci.centos.org/job/fedora-rawhide-build-pipeline/6515/console
13:22:19 Starting nvr-verify [Pipeline] echo 13:22:20 running pipeline step: nvr-verify [Pipeline] withCredentials 21:18:35 Cancelling nested steps due to timeout
https://jenkins-continuous-infra.apps.ci.centos.org/job/fedora-f31-build-pipeline/109/console
22:18:34 + awk -F= '{print "env."$1"=\""$2"\""}' /workDir/workspace/fedora-f31-build-pipeline/koji-build/logs/job.props [Pipeline] load [Pipeline] { (/workDir/workspace/fedora-f31-build-pipeline/job.props.groovy) [Pipeline] } [Pipeline] // load [Pipeline] echo 22:18:36 running pipeline step: koji-build [Pipeline] withCredentials 06:17:13 Cancelling nested steps due to timeout
@jbieren I don't know if this could cause the problem, but do we really need this function to have handlePipelineStep and withCredentials?
https://github.com/openshift/contra-lib/blob/master/vars/executeInContainer.groovy#L24
I'd assume they should be called before running executeInContainer, if necessary.
@bgoncalv I wasn't involved with the writing of that function, so I can't speak to its design. However, I am not sure what harm they would cause by being there if they aren't being used. I know it is the last thing you see in your console output when this is hanging, but nothing after it is going to give any output until https://github.com/openshift/contra-lib/blob/master/vars/executeInContainer.groovy#L39 so it is hard to be sure what is hanging
we hit it again:
02:01:39 + awk -F= '{print "env."$1"=\""$2"\""}' /workDir/workspace/fedora-f30-build-pipeline/koji-build/logs/job.props [Pipeline] load [Pipeline] { (/workDir/workspace/fedora-f30-build-pipeline/job.props.groovy) [Pipeline] } [Pipeline] // load [Pipeline] echo 02:01:40 running pipeline step: koji-build [Pipeline] withCredentials 10:00:13 Cancelling nested steps due to timeout [Pipeline] // withCredentials
https://jenkins-continuous-infra.apps.ci.centos.org/job/fedora-f30-build-pipeline/924/console
Let's see if https://github.com/CentOS-PaaS-SIG/upstream-fedora-pipeline/pull/185 fixes the issue
@jbieren the PR above seems to solve the issue, but we still run some functions the depend on executeInContainer from contra-lib, such as the functions from the jobs to test qcow2 https://github.com/CentOS-PaaS-SIG/upstream-fedora-pipeline/blob/cde23212a27e843c082e8d93a6e60f7ba7ae8107/vars/fedoraImageTest.groovy would be possible to change the contra-lib function? Otherwise I will need to update the job that validates the qcow2.
executeInContainer
@bgoncalv I am not the only person who works with contra-lib. You are welcome to raise a PR to the repo to make the change you want to executeInContainer, and it will need approval from others (not just me) to be accepted and merged. The library was created to be used by numerous teams, and I believe other teams are using the executeInContainer function, so we cannot break them
Metadata Update from @bgoncalv: - Issue status updated to: Closed (was: Open)