freeipa

#8654 DNSSEC key synchronization issues
Closed: fixed by frenaud. Opened by abbra.

Closed: fixed
Reopen Issue

As reported on freeipa-users@, there are some edge cases that current ipa-ods-exporter code cannot handle.

  1. Master key is missing in the local HSM: 
Jan 10 08:07:00 hn-dlp systemd[1]: Started IPA OpenDNSSEC Signer replacement.
Jan 10 08:07:03 hn-dlp platform-python[2831]: Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO
Jan 10 08:07:03 hn-dlp platform-python[2831]: Configuration.cpp(96): Missing slots.mechanisms in configuration. Using default value: ALL
Jan 10 08:07:03 hn-dlp platform-python[2831]: Configuration.cpp(124): Missing slots.removable in configuration. Using default value: false
Jan 10 08:07:03 hn-dlp /usr/libexec/ipa/ipa-ods-exporter[2831]: new replica keys in LDAP: set()
Jan 10 08:07:03 hn-dlp /usr/libexec/ipa/ipa-ods-exporter[2831]: obsolete replica keys in local HSM: set()
Jan 10 08:07:03 hn-dlp /usr/libexec/ipa/ipa-ods-exporter[2831]: ldap2master_replica: keys in local HSM & LDAP: {'0xbb3d905d65b56247a74eeda39285b6b6', '0x699c52466073aba4c50cfb80a2328204', '0x8cd9a2579a1b2f741ad3420ed0291a18'}
Jan 10 08:07:03 hn-dlp ipa-ods-exporter[2831]: Traceback (most recent call last):
Jan 10 08:07:03 hn-dlp ipa-ods-exporter[2831]:  File "/usr/libexec/ipa/ipa-ods-exporter", line 702, in <module>
Jan 10 08:07:03 hn-dlp ipa-ods-exporter[2831]:    master2ldap_master_keys_sync(ldapkeydb, localhsm)
Jan 10 08:07:03 hn-dlp ipa-ods-exporter[2831]:  File "/usr/libexec/ipa/ipa-ods-exporter", line 378, in master2ldap_master_keys_sync
Jan 10 08:07:03 hn-dlp ipa-ods-exporter[2831]:    mkey_local = localhsm.find_keys(id=mkey_id).popitem()[1]
Jan 10 08:07:03 hn-dlp ipa-ods-exporter[2831]: KeyError: 'popitem(): dictionary is empty'
Jan 10 08:07:03 hn-dlp systemd[1]: ipa-ods-exporter.service: Main process exited, code=exited, status=1/FAILURE
Jan 10 08:07:03 hn-dlp systemd[1]: ipa-ods-exporter.service: Failed with result 'exit-code'.
  1. Inability to skip keys that already exist during their import:
Jan 10 08:35:02 hn-dlp platform-python[4152]: Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO
Jan 10 08:35:02 hn-dlp platform-python[4152]: Configuration.cpp(96): Missing slots.mechanisms in configuration. Using default value: ALL
Jan 10 08:35:02 hn-dlp platform-python[4152]: Configuration.cpp(124): Missing slots.removable in configuration. Using default value: false
Jan 10 08:35:02 hn-dlp /usr/libexec/ipa/ipa-ods-exporter[4152]: new replica keys in LDAP: {'0x8cd9a2579a1b2f741ad3420ed0291a18', '0xbb3d905d65b56247a74eeda39285b6b6', '0x699c52466073aba4c50cfb80a2328204'}
Jan 10 08:35:02 hn-dlp ipa-ods-exporter[4152]: Traceback (most recent call last):
Jan 10 08:35:02 hn-dlp ipa-ods-exporter[4152]:  File "/usr/libexec/ipa/ipa-ods-exporter", line 701, in <module>
Jan 10 08:35:02 hn-dlp ipa-ods-exporter[4152]:    ldap2master_replica_keys_sync(ldapkeydb, localhsm)
Jan 10 08:35:02 hn-dlp ipa-ods-exporter[4152]:  File "/usr/libexec/ipa/ipa-ods-exporter", line 304, in ldap2master_replica_keys_sync
Jan 10 08:35:02 hn-dlp ipa-ods-exporter[4152]:    localhsm.import_public_key(new_key_ldap, new_key_ldap['ipapublickey'])
Jan 10 08:35:02 hn-dlp ipa-ods-exporter[4152]:  File "/usr/lib/python3.6/site-packages/ipaserver/dnssec/localhsm.py", line 183, in import_public_key
Jan 10 08:35:02 hn-dlp ipa-ods-exporter[4152]:    h = self.p11.import_public_key(**params)
Jan 10 08:35:02 hn-dlp ipa-ods-exporter[4152]:  File "/usr/lib/python3.6/site-packages/ipaserver/p11helper.py", line 1451, in import_public_key
Jan 10 08:35:02 hn-dlp ipa-ods-exporter[4152]:    raise DuplicationError("Public key with same ID already exists")
Jan 10 08:35:02 hn-dlp ipa-ods-exporter[4152]: ipaserver.p11helper.DuplicationError: Public key with same ID already exists
Jan 10 08:35:03 hn-dlp systemd[1]: ipa-ods-exporter.service: Main process exited, code=exited, status=1/FAILURE
Jan 10 08:35:03 hn-dlp systemd[1]: ipa-ods-exporter.service: Failed with result 'exit-code'.
  1. The reason for the above seems to be a missing master key material in LDAP as ipaSecretKeyRefObject object class is missing as there is no ipaSecretKeyRef attribute and we are unable to unwrap the key: 
Jan 10 08:43:13 hn-dlp ipa-dnskeysyncd[5557]: ipapython.ipautil.CalledProcessError: CalledProcessError(Command ['/usr/libexec/ipa/ipa-dnskeysync-replica'] returned non-zero exit status 1: '
....
ipa-dnskeysync-replica: DEBUG    master keys in local HSM: {\'0x218ea9686a3c2ae9bcad788d1c412dfd\', \'0x295cecf0ebb2c197928086c33c4b01ec\', \'0x7038dd6e887591694c48d05f0a73c87a\', \'0x302250bcd4461cbd5e5da827fa59de45\',
 \'0x57d2d80e09efdb200ff7a406f7acf6aa\'}
ipa-dnskeysync-replica: DEBUG    master keys in LDAP HSM: {\'0x3e0cafd58625e3490b7650028f979d02\', \'0x295cecf0ebb2c197928086c33c4b01ec\', \'0x57d2d80e09efdb200ff7a406f7acf6aa\'}
ipa-dnskeysync-replica: DEBUG new master keys in LDAP HSM: {\'0x3e0cafd58625e3490b7650028f979d02\'}
Traceback (most recent call last):
  File "/usr/libexec/ipa/ipa-dnskeysync-replica", line 189, in <module>
    ldap2replica_master_keys_sync(ldapkeydb, localhsm)
  File "/usr/libexec/ipa/ipa-dnskeysync-replica", line 80, in ldap2replica_master_keys_sync
    str_hexlify(mkey_id)
ValueError: Master key 0x3e0cafd58625e3490b7650028f979d02 in LDAP is missing key material referenced by ipaSecretKeyRefObject attribute
')
Jan 10 08:43:14 hn-dlp systemd[1]: ipa-dnskeysyncd.service: Main process exited, code=exited, status=1/FAILURE
Jan 10 08:43:14 hn-dlp systemd[1]: ipa-dnskeysyncd.service: Failed with result 'exit-code'.
Jan 10 08:44:03 hn-dlp systemd[1]: ipa-ods-exporter.service: Service RestartSec=1min expired, scheduling restart.
Jan 10 08:44:03 hn-dlp systemd[1]: ipa-ods-exporter.service: Scheduled restart job, restart counter is at 45.
Jan 10 08:44:03 hn-dlp systemd[1]: Stopped IPA OpenDNSSEC Signer replacement.

The first issue (missing master key in localhsm) can happen with the following scenario:
- install ipa server with dns on server1
- install replica (no dns) on server2
- enable dnssec on server1, disable dnssec on server1
- install dns + dnssec on server2

Since the master key generated on server1 hasn't been used to sign anything, and a new master key is generated on server2, the missing master key can be ignored.

The 2nd issue (already existing key) is more difficult to trigger. When dnssec is disabled on the local node, a new replica key is created and the older replica keys are marked as disabled in the localhsm and in LDAP with ipk11wrap=False. If ipa-ods-enforcer runs in the middle of this operation, with the key already marked as disabled in localhsm but not yet in LDAP, ipa-ods-enforcer believes there is a new replica key in LDAP that needs to be synchronized to the localhsm but the sync fails because the key is already present. This error can also be ignored.

Metadata Update from @frenaud:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/5612
- Issue assigned to frenaud

master:

  • 3b1a56f588b7585bb98440454f2db72360233bb4 dnssec: fix ipa-ods-exporter crash when master key missing
  • 73ced07e2eefee137716452198d7cb0d59016c8f dnssec: concurrency issue when disabling old replica key

ipa-4-9:

  • 9cfcbc67e12c2ce6e98d3ed1645bc592a573199e dnssec: fix ipa-ods-exporter crash when master key missing
  • 7616f1dad2b478c5e4f850587da1bd4e52108fa1 dnssec: concurrency issue when disabling old replica key

ipa-4-8:

  • 040981c327a5175c94407eab3b9170e98da5fbfe dnssec: fix ipa-ods-exporter crash when master key missing
  • 03f5413604f9504626e259c10168dafa2302d1cb dnssec: concurrency issue when disabling old replica key

Metadata Update from @frenaud:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)
Metadata
None
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.