#111 Timeout is too short for running CI tests for gcc
Closed by bgoncalv. Opened by tstellar.

I'm trying to add CI tests for the gcc package, but gcc takes more than 8 hours to build, so the CI test fails due to timeout before any tests can be run:

Here is an example:
https://jenkins-continuous-infra.apps.ci.centos.org/job/fedora-rawhide-pr-pipeline/3501/


How long does gcc generally take to run it's tests? I assume the quick fix is to extend the timeout for everything, but we may need to make some special timeouts for larger packages to avoid waiting too long on smaller ones.

The tests should take only a few minutes to run. The problem is that the scratch build takes over 8 hours, so there is no chance to even start the tests.

@msrb @bgoncalv is there a way we can delay the test until after the build is done instead of waiting for long builds? If not, does it make sense to have a table of long builds to allow to run longer?

@tstellar @jimbair we could increase the pipeline timeout when running pull requests, but the question is how long should it be...

Checking https://koji.fedoraproject.org/koji/taskinfo?taskID=44419848 it is running for over 40 hours...

Yeah, I am afraid there is no universal right answer here. Instead of setting the timeout globally for the whole pipeline, we could set timeouts for individual stages. And the build stage would have some extra long timeout.

@tstellar @jimbair we could increase the pipeline timeout when running pull requests, but the question is how long should it be...
Checking https://koji.fedoraproject.org/koji/taskinfo?taskID=44419848 it is running for over 40 hours...

This build looks like it is stuck. It seems like there may be an issue with the gcc builds. Looking at the history, the recent builds have all been ~7 hours, but a few months ago they were completing in ~2 hrs. Maybe we can do something to bring the gcc build times back down, so we don't need to adjust the timeouts.

@jimbair @msrb I've created https://github.com/CentOS-PaaS-SIG/upstream-fedora-pipeline/pull/226/ that increases the time out to 12hrs, this should help with most of scratch builds.

@bgoncalv @msrb to @tstellar's point above, do we know why builds went up from ~2 hours to ~7 hours? Is it possible to fix the long build times instead?

@bgoncalv @msrb to @tstellar's point above, do we know why builds went up from ~2 hours to ~7 hours? Is it possible to fix the long build times instead?

This time difference has nothing to do with the pipeline, it is the build in koji that is taking longer...

@tstellar are you still hitting the timeout issue? We've increased the pipeline timeout to 12hr.

I'm closing this ticket, but if koji scratch builds are still taking that long we might need to ask someone from koji team to check why the builds are taking so long to finish.

Metadata Update from @bgoncalv:
- Issue status updated to: Closed (was: Open)

@bgoncalv My pull request hasn't been merged yet, so I haven't tried gcc tests again, but it looks like the last gcc build in koji was down to around ~5 hours, so it should work.

Metadata