#8611 [Tracker] Nightly failure during replica installation: topo segment not created
Closed: fixed by frenaud. Opened by frenaud.

The nightly test test_topology_TestTopologyOptions is failing with the nightly copr repo for 389ds (@389ds/389-ds-base-nightly), see PR #574 (logs, report):

self = <ipatests.test_integration.test_topology.TestTopologyOptions object at 0x7fb0c4d13c70>
    def test_topology_updated_on_replica_install_remove(self):
        """
        Install and remove a replica and make sure topology information is
        updated on all other replicas
        Testcase: http://www.freeipa.org/page/V4/Manage_replication_topology/
        Test_plan#Test_case:
        _Replication_topology_should_be_saved_in_the_LDAP_tree
        """
        tasks.kinit_admin(self.master)
>       result1 = self.master.run_command(['ipa', 'topologysegment-find',
                                           DOMAIN_SUFFIX_NAME]).stdout_text
test_integration/test_topology.py:79: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <ipatests.pytest_ipa.integration.host.Host master.ipa.test (master)>
argv = ['ipa', 'topologysegment-find', 'domain'], set_env = True
stdin_text = None, log_stdout = True, raiseonerr = True, cwd = None, bg = False
encoding = 'utf-8', ok_returncode = 0
    def run_command(self, argv, set_env=True, stdin_text=None,
                    log_stdout=True, raiseonerr=True,
                    cwd=None, bg=False, encoding='utf-8', ok_returncode=0):
        """Wrapper around run_command to log stderr on raiseonerr=True
        :param ok_returncode: return code considered to be correct,
                              you can pass an integer or sequence of integers
        """
        result = super().run_command(
            argv, set_env=set_env, stdin_text=stdin_text,
            log_stdout=log_stdout, raiseonerr=False, cwd=cwd, bg=bg,
            encoding=encoding
        )
        # in FIPS mode SSH may print noise to stderr, remove the string
        # "FIPS mode initialized" + optional newline.
        result.stderr_bytes = FIPS_NOISE_RE.sub(b'', result.stderr_bytes)
        try:
            result_ok = result.returncode in ok_returncode
        except TypeError:
            result_ok = result.returncode == ok_returncode
        if not result_ok and raiseonerr:
            result.log.error('stderr: %s', result.stderr_text)
>           raise subprocess.CalledProcessError(
                result.returncode, argv,
                result.stdout_text, result.stderr_text
            )
E           subprocess.CalledProcessError: Command '['ipa', 'topologysegment-find', 'domain']' returned non-zero exit status 1.
pytest_ipa/integration/host.py:200: CalledProcessError

The output of ipa topologysegment-find is the following:

[ipatests.pytest_ipa.integration.host.Host.master.cmd58] RUN ['ipa', 'topologysegment-find', 'domain']
[ipatests.pytest_ipa.integration.host.Host.master.cmd58] ------------------
[ipatests.pytest_ipa.integration.host.Host.master.cmd58] 0 segments matched
[ipatests.pytest_ipa.integration.host.Host.master.cmd58] ------------------
[ipatests.pytest_ipa.integration.host.Host.master.cmd58] ----------------------------
[ipatests.pytest_ipa.integration.host.Host.master.cmd58] Number of entries returned 0
[ipatests.pytest_ipa.integration.host.Host.master.cmd58] ----------------------------
[ipatests.pytest_ipa.integration.host.Host.master.cmd58] Exit code: 1

Currently investigating with Thierry.
The issue appeared with 389-ds-base-2.0.1-20201206gitdec149b4c.fc32.x86_64 but was not present with 389-ds-base-2.0.1-20201128gitce7beae4d.fc32.x86_64. Only 9 commits happened on 389ds between the 2 builds.

I also tried with 389-ds-base-2.0.1-20201202git7d2f95dc5.fc32.x86_64 and the issue also happens with this build.
In summary the problem was introduced in one of these commits: (more recent top)
- 7d2f95d Issue 4243 - Fix test: SyncRepl plugin provides a wrong cookie
- b721951Issue 4464 - RFE - clang with ds+asan+rust
- 782e6c1Issue 4105 - Remove python.six (fix regression)
- a5029c8Issue 4384 - Use MONOTONIC clock for all timing events and conditions
- a98fe54Issue 4418 - ldif2db - offline. Warn the user of skipped entries
- f1243f7 Issue 4243 - Fix test: SyncRepl plugin provides a wrong cookie (#4467)

Companion issue on 389-ds side: https://github.com/389ds/389-ds-base/issues/4478

Latest run was green, see PR 638:
- test_topology_TestTopologyOptions report using 389-ds-base-2.0.1-20210109git65678bb3b.fc32.x86_64

The issue can be closed as fixed.

(Updated) for full ref, the commit on 389ds is https://github.com/389ds/389-ds-base/commit/65678bb3b952123e151cab6211259b4e54d6dd1f

Metadata Update from @frenaud:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Note: issue present in 389-ds 1.4.4.9 but fixed in 1.4.4.10

The update 389-ds-base-1.4.4.10-1.fc33 was unpushed, https://bodhi.fedoraproject.org/updates/FEDORA-2021-e81d94692a

Metadata Update from @frenaud:
- Issue status updated to: Open (was: Closed)

Related failures in other tests:
- test_backup_and_restore.py::TestReplicaInstallAfterRestore::test_replica_install_after_restore
- test_crlgen_manage.py::TestCRLGenManage::test_uninstall_with_ignore_last_of_role
- test_replica_promotion.py::TestWrongClientDomain::test_client_rollback
- test_server_del.py::TestServerDel::test_removal_of_nonexistent_master_raises_error
- test_topology.py::TestCASpecificRUVs::test_replica_uninstall_deletes_ruvs

Another failure: PR680, report and another report

389ds released 1.4.4.11-1.fc33 and 2.0.2-1.fc34 with the fix, closing as fixed.

Metadata Update from @frenaud:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata