About a week ago or so, I started noticing that some my koji builds started to take a lot of time, and ppc64le always was the slowest build task, which struck me as odd, but they eventually finished, so I didn't think too much of it.
But today, I needed to cancel and resubmit three builds because they never un-stuck themselves on ppc64le. It looks like they got stuck when dnf installed build dependencies, either after "Downloading packages" or after "Running transaction".
Examples:
Both tasks which never un-stuck themselves were on buildvm-ppc64le-23.iad2.fedoraproject.org, and the task that took ages on ppc64le but eventually un-stuck itself was on buildvm-ppc64le-24.iad2.fedoraproject.org, if that helps narrow down the issue.
Looking back at my recent builds, it appears that ppc64le is now often the slowest architecture, even slower than armv7hl, in almost all my recent builds ... so something is going on.
So, I think this might be related to my messing with the i/o scheduler on those vmhosts. :(
I've switched it back to the default scheduler... can you see if this is any better now?
Thanks, I'll keep an eye on it, I'll probably have to do some builds over the weekend and will report back if there's still anything odd going on.
Metadata Update from @zlopez: - Issue priority set to: Waiting on Reporter (was: Needs Review) - Issue tagged with: low-gain, low-trouble, ops
I've now built over 100 packages in koji over the last two days, and none of them got stuck, so looks like fiddling with the I/O scheduler was indeed the problem. Thanks! :)
Metadata Update from @decathorpe: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.