freeipa

#8023 [Tracker] Test failure in test_integration/test_topology.py::TestCASpecificRUVs::()::test_replica_uninstall_deletes_ruvs
Opened by frenaud. Modified

Open
Close Issue

Nightly failure detected in [testing_f29] in the test fedora-29/test_topology_TestCASpecificRUVs, see PR 3442.

The test is failing in test_integration/test_topology.py::TestCASpecificRUVs::()::test_replica_uninstall_deletes_ruvs.

self = <ipatests.test_integration.test_topology.TestCASpecificRUVs object at 0x7fb643ff1ef0>
    def test_replica_uninstall_deletes_ruvs(self):
        """
            http://www.freeipa.org/page/V4/Manage_replication_topology_4_4/Test_Plan
            #Test_case:_.2A-ruv_subcommands_of_ipa-replica-manage_are_extended
            _to_handle_CA-specific_RUVs
            """
        master = self.master
        replica = self.replicas[1]
        res1 = master.run_command(['ipa-replica-manage', 'list-ruv', '-p',
                                  master.config.dirman_password]).stdout_text
        assert(res1.count(replica.hostname) == 2), (
            "Did not find proper number of replica hostname (%s) occurrencies"
            " in the command output: %s" % (replica.hostname, res1))
        master.run_command(['ipa-replica-manage', 'del', replica.hostname,
                            '-p', master.config.dirman_password])
        tasks.uninstall_master(replica)
        # ipa-replica-manage del launches a clean-ruv task which is
        # ASYNCHRONOUS
        # wait for the task to finish before checking list-ruv
        tasks.wait_for_cleanallruv_tasks(self.master.ldap_connect())
        res2 = master.run_command(['ipa-replica-manage', 'list-ruv', '-p',
                                  master.config.dirman_password]).stdout_text
>       assert(replica.hostname not in res2), (
            "Replica RUVs were not clean during replica uninstallation")
E       AssertionError: Replica RUVs were not clean during replica uninstallation
E       assert 'replica0.ipa.test' not in 'Replica Update V...ipa.test:389: 6\n'
E         'replica0.ipa.test' is contained here:
E           Replica Update Vectors:
E               master.ipa.test:389: 4
E               replica1.ipa.test:389: 3
E               replica0.ipa.test:389: 7
E           Certificate Server Replica Update Vectors:
E               master.ipa.test:389: 6

Test scenario (test_delete_ruvs):
- install master + 2 replicas
- stop ds on replica0
- on replica0 call ipa-replica-manage clean-ruv -f twice (to clear the ruv on CA suffix and on domain suffix)
- check that there is no ruv any more with replica0
- start ds on replica0
Then second test (test_replica_uninstall_deletes_ruvs):
- on master run ipa-replica-manage del replica1
- on replica1 ipa-server-install --uninstall
- on master wait for cleanallruv task to finish
- on master check that list-ruv does not show any more replica1

The test fails at the last step.

Note: the code for wait_for_cleanallruv_tasks looks buggy as it does not return any exception if the function exits on timeout.

Another failure: PR381, logs

Another failure: PR393, logs

Another failure: PR424, logs

The failure seems to happen more frequently, see PR #483

Another failure seen in PR522, logs.

However, it has not been present for some time (approx two months)

389ds error log on the master show ERR - ipa-topology-plugin - ipa_topo_util_cleanruv: failed to create cleanalltuv task, opening a companion issue against 389ds:
https://github.com/389ds/389-ds-base/issues/4442

Metadata Update from @frenaud:
- Issue tagged with: tracker

Another failure observed: PR559, logs

Also happened in PR #574 on [testing_master_389ds] = fedora32 with 389ds nightly copr repo.

Another failure in [testing_master_pki] Nightly PR #717 log
Edited by amore

Another failure in [testing_master_pki] Nightly PR #732 report

Failure observed in [testing_ipa-4.9_latest_selinux] Nightly PR #739 but not in non-selinux counterpart. report

Another failure in [testing_master_pki] Nightly PR #762 report

Another failure observed in selinux environment - [testing_ipa-4.9_latest_selinux] Nightly PR #754 , report

Failiure observed in testing_master_previous

PR #784
Logs

Failure observed in [testing_master_pki] Nightly PR #849 report

Failed in [testing_master_latest_selinux]

PR #959

report

Failed in [testing_master_latest_selinux]

PR #974

report

Failure observed in testing_master_previous PR 1057 Logs

failure observed in testing_master_previous PR 1087 Logs

failed in [testing_master_testing Nightly PR #1084](https://github.com/freeipa-pr-ci2/freeipa/pull/1084 logs

failure observed in [testing_master_pki] Nightly PR#1095
report

Failed in [testing_master_pki]
PR #1199
report

Failed in [testing_master_pki]
PR #1446
report

Failed in testing_master_testing_selinux PR 1483 report

failure seen intesting_master_latest PR 1487 Report

failure seen in testing_master_latest PR 1584 Report

test failure seen in testing_master_latest PR 1592 Report

test failure seen in testing_master_latest PR 1608 Report

test failure reproducible in testing_master_latest PR 1838 Report

Reproducible in testing_master_latest PR 1856 Report

Reproducible in testing_master_pki Report

Reproducible in testing_master_389ds PR 1891 Report

Reproducible in testing_master_latest PR 1975 Report

Any chance a fresh copy of the DS logs could be provided? Previously links are returning error 404...

@mreynolds
Sure, here you go: http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/1fabcc56-341c-11ed-801a-fa163e67bee2/

From there you can navigate to the test logs (http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/1fabcc56-341c-11ed-801a-fa163e67bee2/test_integration-test_topology.py-TestCASpecificRUVs-test_delete_ruvs/), where you can find the a folder for each machine, containing all the collected logs.

Reproduced in testing_master_pki
report

@mreynolds
Sure, here you go: http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/1fabcc56-341c-11ed-801a-fa163e67bee2/

From there you can navigate to the test logs (http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/1fabcc56-341c-11ed-801a-fa163e67bee2/test_integration-test_topology.py-TestCASpecificRUVs-test_delete_ruvs/), where you can find the a folder for each machine, containing all the collected logs.

@frenaud I'm not seeing any errors in the DS logs. All CleanAllRUV tasks completed successfully. I don't see any errors about not being able to update the task entry either.

If this was from a "failed" test I wonder if it's simply a timing issue? Perhaps the freeipa test is running too quickly and not giving enough time to the cleanallruv task to finish? I can see in the logs that it finishes successfully for rids 5 and 3.

Maybe we need a sleep in tasks.wait_for_cleanallruv_tasks() ? I'm not sure how this function works though...
Metadata
None

test-failure tests tracker

None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.