The Sigul signing queue on sign-vault01 was blocked when messages for all subpackages and architectures of helix-25.07.1-3.fc42 were queued for signing after the build was tagged. During processing, Sigul reported:
Request error: The RPM file is corrupt nss.error.NSPRError: (SEC_ERROR_LIBRARY_FAILURE) security library failure
The issue caused the signing queue to stall. The affected RPMs were removed from the RabbitMQ signing queue, after which Sigul resumed normal operation.
ASAP
Fedora cease to exist?
Please check https://www.fedorastatus.org/ for any known outages before filing issues on an outage.
@mikem, can you look into what happened here?
Metadata Update from @jnsamyak: - Issue tagged with: medium-gain, medium-trouble, ops
Specifically, it's the helix-debuginfo-25.07.1-3.fc42.ppc64le that looks to be failing a digest check:
❯ rpm --nosignature -K helix*.rpm helix-25.07.1-3.fc42.aarch64.rpm: digests OK helix-25.07.1-3.fc42.ppc64le.rpm: digests OK helix-25.07.1-3.fc42.s390x.rpm: digests OK helix-25.07.1-3.fc42.src.rpm: digests OK helix-25.07.1-3.fc42.x86_64.rpm: digests OK helix-debuginfo-25.07.1-3.fc42.aarch64.rpm: digests OK helix-debuginfo-25.07.1-3.fc42.ppc64le.rpm: DIGESTS NOT OK helix-debuginfo-25.07.1-3.fc42.s390x.rpm: digests OK helix-debuginfo-25.07.1-3.fc42.x86_64.rpm: digests OK helix-debugsource-25.07.1-3.fc42.aarch64.rpm: digests OK helix-debugsource-25.07.1-3.fc42.ppc64le.rpm: digests OK helix-debugsource-25.07.1-3.fc42.s390x.rpm: digests OK helix-debugsource-25.07.1-3.fc42.x86_64.rpm: digests OK
Presumably the build system will try its best to never OK bad packages, but robosignatory should also gracefully handle this, possibly by dead-lettering the message for later investigation.
That file is quite a bit smaller than the corresponding debuginfo RPMs for other architectures, or for ppc64le in other Fedora releases. I suppose it was most likely truncated in some kind of infrastructure glitch.
ppc64le
It seems like the best thing I could do at this point would be to bump the release number and do a fresh build.
This seems ot be a viable solution.
It looks like the replacement build and update are OK. Let me know if you see anything else that I need to do.