#13055 Sigul signing queue blocked by corrupt helix-25.07.1-3.fc42 build (all subpackages/arches)
Opened by jnsamyak. Modified

  • Describe the issue

The Sigul signing queue on sign-vault01 was blocked when messages for all subpackages and architectures of helix-25.07.1-3.fc42 were queued for signing after the build was tagged.
During processing, Sigul reported:

Request error: The RPM file is corrupt
nss.error.NSPRError: (SEC_ERROR_LIBRARY_FAILURE) security library failure

The issue caused the signing queue to stall. The affected RPMs were removed from the RabbitMQ signing queue, after which Sigul resumed normal operation.

  • When do you need this? (YYYY/MM/DD)

ASAP

  • When is this no longer needed or useful? (YYYY/MM/DD)

Fedora cease to exist?

Please check https://www.fedorastatus.org/ for any known
outages before filing issues on an outage.


@mikem, can you look into what happened here?

Metadata Update from @jnsamyak:
- Issue tagged with: medium-gain, medium-trouble, ops

Specifically, it's the helix-debuginfo-25.07.1-3.fc42.ppc64le that looks to be failing a digest check:

❯ rpm --nosignature -K helix*.rpm
helix-25.07.1-3.fc42.aarch64.rpm: digests OK
helix-25.07.1-3.fc42.ppc64le.rpm: digests OK
helix-25.07.1-3.fc42.s390x.rpm: digests OK
helix-25.07.1-3.fc42.src.rpm: digests OK
helix-25.07.1-3.fc42.x86_64.rpm: digests OK
helix-debuginfo-25.07.1-3.fc42.aarch64.rpm: digests OK
helix-debuginfo-25.07.1-3.fc42.ppc64le.rpm: DIGESTS NOT OK
helix-debuginfo-25.07.1-3.fc42.s390x.rpm: digests OK
helix-debuginfo-25.07.1-3.fc42.x86_64.rpm: digests OK
helix-debugsource-25.07.1-3.fc42.aarch64.rpm: digests OK
helix-debugsource-25.07.1-3.fc42.ppc64le.rpm: digests OK
helix-debugsource-25.07.1-3.fc42.s390x.rpm: digests OK
helix-debugsource-25.07.1-3.fc42.x86_64.rpm: digests OK

Presumably the build system will try its best to never OK bad packages, but robosignatory should also gracefully handle this, possibly by dead-lettering the message for later investigation.

Specifically, it's the helix-debuginfo-25.07.1-3.fc42.ppc64le that looks to be failing a digest check:

That file is quite a bit smaller than the corresponding debuginfo RPMs for other architectures, or for ppc64le in other Fedora releases. I suppose it was most likely truncated in some kind of infrastructure glitch.

It seems like the best thing I could do at this point would be to bump the release number and do a fresh build.

It seems like the best thing I could do at this point would be to bump the release number and do a fresh build.

This seems ot be a viable solution.

It looks like the replacement build and update are OK. Let me know if you see anything else that I need to do.

Metadata