#9725 A slow HSM can cause IPA server installation to fail setting up certificate tracking
Closed: fixed by frenaud. Opened by rcritten.

Cloned from https://issues.redhat.com/browse/RHEL-72580

What were you trying to do that didn't work?

Installing an IPA server against a slow HSM may cause installation to fail with:

[21/32]: configure certificate renewals
[error] DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

How reproducible is this bug?:

It depends on the speed of the HSM. certmonger needs to access the token to verify that the requested certificate is available. There is a very short 5 second timeout on this operation.

Here is a candidate fix. I don't think this needs to be user-configurable.

--- /tmp/certmonger.py  2025-01-02 15:22:13.354386507 -0500
+++ /usr/lib/python3.9/site-packages/ipalib/install/certmonger.py       2025-01-02 15:22:42.319386507 -0500
@@ -477,7 +477,7 @@
         request_parameters['cert-perms'] = perms[0]
         request_parameters['key-perms'] = perms[1]
-    result = cm.obj_if.add_request(request_parameters)
+    result = cm.obj_if.add_request(request_parameters, timeout=30)
     try:
         if result[0]:
             request = _cm_dbus_object(cm.bus, cm, result[1], DBUS_CM_REQUEST_IF,
@@ -581,7 +581,7 @@
     if nss_user:
         params['nss-user'] = nss_user
-    result = cm.obj_if.add_request(params)
+    result = cm.obj_if.add_request(params, timeout=30)
     try:
         if result[0]:
             request = _cm_dbus_object(cm.bus, cm, result[1], DBUS_CM_REQUEST_IF,

I should mention that manually starting tracking on the caSigningCert cert-pki-ca certificate once installation has failed took 18 seconds. This is why I'm going to propose waiting up to 30.

Metadata Update from @rcritten:
- Custom field rhbz adjusted to https://issues.redhat.com/browse/RHEL-72580

https://github.com/freeipa/freeipa/pull/7642

Metadata Update from @rcritten:
- Custom field rhbz adjusted to https://issues.redhat.com/browse/RHEL-72580 https://issues.redhat.com/browse/RHEL-73022 (was: https://issues.redhat.com/browse/RHEL-72580)

master:

  • 4776a8babdd25b8fa1afa7e826fd8d153b90f31e Add 30-second timeout for certmonger request/start tracking

ipa-4-12:

  • 2506d5de5a9dd8ebe6efc777c2eb76461f5b57e2 Add 30-second timeout for certmonger request/start tracking

Metadata Update from @frenaud:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata Update from @rcritten:
- Custom field affects_doc adjusted to on
- Custom field knownissue adjusted to on
- Issue status updated to: Open (was: Closed)

We saw additional failures. One certificate required exactly 30 seconds to start tracking and another 35. As a test I bumped the timeout to 5 minutes and was able to install successfully.

During the review ab asked about making this configurable. I resisted because this is such a corner case but since it can prevent installation I'll go ahead and increase the default and see how much work it is to add it to the api env.

The timeout is per DBus call so I'm reluctant to set it too high because theoretically it could add significant time to the installation. I don't know if failing is any better. It'll be a difficult balance to find.

master:

  • c5300a312775676ce64a3aac3cde2d83ae5f2fde Apply certmonger_timeout to start_tracking and request_cert

ipa-4-12:

  • 9f30edef463237ba48efe45406626eb325bf6c39 Apply certmonger_timeout to start_tracking and request_cert

Metadata Update from @frenaud:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata