freeipa

#8592 Nightly test failure in test_uninstallation.py::TestUninstallBase::test_failed_uninstall
Opened by frenaud. Modified

Open
Close Issue

The nightly test test_uninstallation.py::TestUninstallBase::test_failed_uninstall failed in the master branch, see PR #542.
Logs and report:

self = <ipatests.test_integration.test_uninstallation.TestUninstallBase object at 0x7f93df736c10>
    def test_failed_uninstall(self):
        self.master.run_command(['ipactl', 'stop'])
        serverid = realm_to_serverid(self.master.domain.realm)
        instance_name = ''.join([DS_INSTANCE_PREFIX, serverid])
        try:
            # Moving the DS instance out of the way will cause the
            # uninstaller to raise an exception and return with a
            # non-zero return code.
            self.master.run_command([
                '/bin/mv',
                '%s/%s' % (paths.ETC_DIRSRV, instance_name),
                '%s/%s.test' % (paths.ETC_DIRSRV, instance_name)
            ])
            cmd = self.master.run_command([
                'ipa-server-install',
                '--uninstall', '-U'],
                raiseonerr=False
            )
            assert cmd.returncode == 1
        finally:
            # Be paranoid. If something really went wrong then DS may
            # be marked as uninstalled so server cert will still be
            # tracked and the instances may remain. This can cause
            # subsequent installations to fail so be thorough.
            dashed_domain = self.master.domain.realm.replace(".", '-')
            dirsrv_service = "dirsrv@%s.service" % dashed_domain
            self.master.run_command(['systemctl', 'stop', dirsrv_service])
            # Moving it back should allow the uninstall to finish
            # successfully.
            self.master.run_command([
                '/bin/mv',
                '%s/%s.test' % (paths.ETC_DIRSRV, instance_name),
                '%s/%s' % (paths.ETC_DIRSRV, instance_name)
            ])
            cmd = self.master.run_command([
                'ipa-server-install',
                '--uninstall', '-U'],
                raiseonerr=False
            )
            assert cmd.returncode == 0
>           self.master.run_command([
                paths.IPA_GETCERT,
                'stop-tracking',
                '-d', '%s/%s' % (paths.ETC_DIRSRV, instance_name),
                '-n', 'Server-Cert',
            ])

The getcert stop-tracking command fails with:

Path "/etc/dirsrv/slapd-IPA-TEST": No such file or directory.

Test scenario:
- stop services
- move the directory /etc/dirsrv/slapd-XXX to /etc/dirsrv/slapd-XXX.test
- uninstall the server, expects failure as the dirsrv instance cannot be found at the expected location
- in the cleanup code, move the instance back to /etc/dirsrv/slapd-XXX, uninstall, call stop-tracking for dirsrv cert.

The error happens in stop-tracking as the directory /etc/dirsrv/slapd-XXX does not exist any more. The test should handle the exception gracefully.

Additional investigations:
the failure is not happening consistently and depends on the root cause of the first un-installation. The sequence during uninstall is the following:
- uninstall CA
- uninstall DS
If the uninstall fails during the step "uninstall CA" because of issue https://pagure.io/freeipa/issue/8506, the code doesn't reach DS uninstallation and leaves the serverid in sysrestore.state file. The subsequent uninstallation is then able to read this serverid and fully uninstall DS, hence untracking the cert + removing the directory.
If the uninstall fails during the step "uninstall DS", the serverid is removed from sysrestore.state. The subsequent uninstallation is unable find the serverid and does not stop tracking the certs.

Code extract from dsinstance.py:

        serverid = self.restore_state("serverid")
        if serverid is not None:
            # What if this fails? Then what?
            self.stop_tracking_certificates(serverid)
            logger.debug("Removing DS instance %s", serverid)
            try:
                remove_ds_instance(serverid)
            except ipautil.CalledProcessError:
                logger.error("Failed to remove DS instance. You may "
                             "need to remove instance data manually")
        else:
            logger.error("Failed to remove DS instance. No serverid present "
                         "in sysrestore file.")

Similar failure observed in PR
report

Unfortunately its been long enough that the logs have been removed.

I wonder if this would be enough to avoid a failure:

if os.path.exists(os.path.join(paths.ETC_DIRSRV, instance_name)):
    <stop tracking>

Unfortunately its been long enough that the logs have been removed.

I wonder if this would be enough to avoid a failure:

if os.path.exists(os.path.join(paths.ETC_DIRSRV, instance_name)):
    <stop tracking>
Metadata
None

test-failure tests

None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.