It seems like https://osci-jenkins-1.ci.fedoraproject.org/ Jenkins is running with only 4 gigs of memory. This was fine last week when the highest load was ~200 concurrent builds, but this week we enabled also rpminspect and dist-git pipelines there. And there were 600 builds running at the same time, which killed the instance (I am not surprised personally).
$ cat /sys/fs/cgroup/memory/memory.limit_in_bytes 4294967296 $
Since 4 gigs of memory is probably a default value from Jenkins helm chart that was used to deploy the instance, it is possible that number of CPUs is also quite low.
Note https://osci-jenkins-2.ci.fedoraproject.org instance is affected by these quite low resources limits as well.
cc @bgoncalv as he has jobs running on the second instance
@astepano @jimbair which one of you fine folks would be able to increase the resource limits on those instances? :)
https://osci-jenkins-2.ci.fedoraproject.org Core: Jenkins 2.235.5
Limits: cpu: 3 memory: 12Gi Requests: cpu: 1 memory: 4Gi
https://osci-jenkins-1.ci.fedoraproject.org/
Closing this ticket.
How I can close this ticket? I cannot modify its metadata.
@astepano you weren't a member of this group, I added you.
Metadata Update from @astepano: - Issue assigned to astepano
@msrb thank you for adding.
Metadata Update from @astepano: - Issue status updated to: Closed (was: Open)