test_replica2_with_dns_install is failing with
test_replica2_with_dns_install
E subprocess.CalledProcessError: Command '['ipa-replica-install', '-w', 'Secret.123', '-p', 'Secret.123', '-U', '--setup-dns', '--forwarder', '192.168.122.1', '--ip-address', '192.168.122.31', '-r', 'IPA.TEST', '--domain', 'ipa.test']' returned non-zero exit status 1.
self = <ipatests.test_integration.test_installation.TestInstallWithCA_DNS2 object at 0x7f4ecc6d9550> def test_replica2_with_dns_install(self): tasks.install_replica(self.master, self.replicas[2], setup_ca=False, > setup_dns=True) test_integration/test_installation.py:127: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ pytest_ipa/integration/tasks.py:444: in install_replica stdin_text=stdin_text) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ipatests.pytest_ipa.integration.host.Host replica1.ipa.test (replica)> argv = ['ipa-replica-install', '-w', 'Secret.123', '-p', 'Secret.123', '-U', ...] set_env = True, stdin_text = None, log_stdout = True, raiseonerr = True cwd = None, ok_returncode = 0 def run_command(self, argv, set_env=True, stdin_text=None, log_stdout=True, raiseonerr=True, cwd=None, ok_returncode=0): """Wrapper around run_command to log stderr on raiseonerr=True :param ok_returncode: return code considered to be correct, you can pass an integer or sequence of integers """ result = super(Host, self).run_command( argv, set_env=set_env, stdin_text=stdin_text, log_stdout=log_stdout, raiseonerr=False, cwd=cwd ) try: result_ok = result.returncode in ok_returncode except TypeError: result_ok = result.returncode == ok_returncode if not result_ok and raiseonerr: result.log.error('stderr: %s', result.stderr_text) raise subprocess.CalledProcessError( result.returncode, argv, > result.stdout_text ) E subprocess.CalledProcessError: Command '['ipa-replica-install', '-w', 'Secret.123', '-p', 'Secret.123', '-U', '--setup-dns', '--forwarder', '192.168.122.1', '--ip-address', '192.168.122.31', '-r', 'IPA.TEST', '--domain', 'ipa.test']' returned non-zero exit status 1. pytest_ipa/integration/host.py:90: CalledProcessError ----------------------------- Captured stderr call ----------------------------- [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/ipareplica-conncheck.log to list of logs to collect [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/ipareplica-ca-install.log to list of logs to collect [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/ipaclient-install.log to list of logs to collect [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/ipaserver-kra-install.log to list of logs to collect [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/ipa-custodia.audit.log to list of logs to collect [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/ipaclient-uninstall.log to list of logs to collect [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/iparestore.log to list of logs to collect [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/ipabackup.log to list of logs to collect [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/kadmind.log to list of logs to collect [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/krb5kdc.log to list of logs to collect [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/httpd/error_log to list of logs to collect [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/pki/ to list of logs to collect [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/audit/audit.log to list of logs to collect [ipatests.pytest_ipa.integration.host.Host.master.ParamikoTransport] RUN ['kinit', 'admin'] [ipatests.pytest_ipa.integration.host.Host.master.cmd36] RUN ['kinit', 'admin'] [ipatests.pytest_ipa.integration.host.Host.master.cmd36] Password for admin@IPA.TEST: [ipatests.pytest_ipa.integration.host.Host.master.cmd36] Exit code: 0 [ipatests.pytest_ipa.integration.host.Host.master.ParamikoTransport] RUN ['ipa', 'dnsconfig-mod', '--allow-sync-ptr=true'] [ipatests.pytest_ipa.integration.host.Host.master.cmd37] RUN ['ipa', 'dnsconfig-mod', '--allow-sync-ptr=true'] [ipatests.pytest_ipa.integration.host.Host.master.cmd37] ipa: ERROR: no modifications to be performed [ipatests.pytest_ipa.integration.host.Host.master.cmd37] Exit code: 1 [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/var/log/ipaclient-install.log to list of logs to collect [ipatests.pytest_ipa.integration.host.Host.replica1.ParamikoTransport] RUN ['true'] [ipatests.pytest_ipa.integration.host.Host.replica1.cmd14] RUN ['true'] [ipatests.pytest_ipa.integration.host.Host.replica1.cmd14] Exit code: 0 [ipatests.pytest_ipa.integration] Adding replica1.ipa.test:/ipatests/env.sh to list of logs to collect [ipatests.pytest_ipa.integration.host.Host.replica1.ParamikoTransport] STAT /ipatests [ipatests.pytest_ipa.integration.host.Host.replica1.ParamikoTransport] WRITE /ipatests/env.sh [ipatests.pytest_ipa.integration.host.Host.replica1.ParamikoTransport] STAT /etc/hostname [ipatests.pytest_ipa.integration.host.Host.replica1.ParamikoTransport] STAT /ipatests/file_backup/etc [ipatests.pytest_ipa.integration.host.Host.replica1.ParamikoTransport] RUN ['cp', '-af', '/etc/hostname', '/ipatests/file_backup/etc/hostname'] [ipatests.pytest_ipa.integration.host.Host.replica1.cmd15] RUN ['cp', '-af', '/etc/hostname', '/ipatests/file_backup/etc/hostname'] [ipatests.pytest_ipa.integration.host.Host.replica1.cmd15] Exit code: 0 [ipatests.pytest_ipa.integration.host.Host.replica1.ParamikoTransport] WRITE /etc/hostname [ipatests.pytest_ipa.integration.host.Host.replica1.ParamikoTransport] RUN ['hostname', 'replica1.ipa.test'] [ipatests.pytest_ipa.integration.host.Host.replica1.cmd16] RUN ['hostname', 'replica1.ipa.test'] [ipatests.pytest_ipa.integration.host.Host.replica1.cmd16] Exit code: 0 [ipatests.pytest_ipa.integration.host.Host.replica1.ParamikoTransport] RUN hostname > '/ipatests/backup_hostname' [ipatests.pytest_ipa.integration.host.Host.replica1.cmd17] RUN hostname > '/ipatests/backup_hostname' [ipatests.pytest_ipa.integration.host.Host.replica1.cmd17] Exit code: 0 [ipatests.pytest_ipa.integration.host.Host.master.ParamikoTransport] RUN ['kinit', 'admin'] [ipatests.pytest_ipa.integration.host.Host.master.cmd38] RUN ['kinit', 'admin'] [ipatests.pytest_ipa.integration.host.Host.master.cmd38] Password for admin@IPA.TEST: [ipatests.pytest_ipa.integration.host.Host.master.cmd38] Exit code: 0 [ipatests.pytest_ipa.integration.host.Host.master.ParamikoTransport] RUN ['ipa', 'dnsconfig-mod', '--allow-sync-ptr=true'] [ipatests.pytest_ipa.integration.host.Host.master.cmd39] RUN ['ipa', 'dnsconfig-mod', '--allow-sync-ptr=true'] [ipatests.pytest_ipa.integration.host.Host.master.cmd39] ipa: ERROR: no modifications to be performed [ipatests.pytest_ipa.integration.host.Host.master.cmd39] Exit code: 1 [ipatests.pytest_ipa.integration.host.Host.master.ParamikoTransport] RUN ['ipa', 'dnszone-add', '122.168.192.in-addr.arpa.'] [ipatests.pytest_ipa.integration.host.Host.master.cmd40] RUN ['ipa', 'dnszone-add', '122.168.192.in-addr.arpa.'] [ipatests.pytest_ipa.integration.host.Host.master.cmd40] ipa: ERROR: DNS zone with name "122.168.192.in-addr.arpa." already exists [ipatests.pytest_ipa.integration.host.Host.master.cmd40] Exit code: 1 ipa: WARNING: ipa: ERROR: DNS zone with name "122.168.192.in-addr.arpa." already exists [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Update in progress, 4 seconds elapsed [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Update succeeded [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [30/42]: prevent time skew after initial replication [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [31/42]: adding sasl mappings to the directory [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [32/42]: updating schema [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [33/42]: setting Auto Member configuration [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [34/42]: enabling S4U2Proxy delegation [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [35/42]: initializing group membership [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [36/42]: adding master entry [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [37/42]: initializing domain level [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [38/42]: configuring Posix uid/gid generation [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [39/42]: adding replication acis [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [40/42]: activating sidgen plugin [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [41/42]: activating extdom plugin [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [42/42]: configuring directory to start on boot [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Done configuring directory server (dirsrv). [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Configuring Kerberos KDC (krb5kdc) [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [1/5]: configuring KDC [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [2/5]: adding the password extension to the directory [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [3/5]: creating anonymous principal [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [4/5]: starting the KDC [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [5/5]: configuring KDC to start on boot [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Done configuring Kerberos KDC (krb5kdc). [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Configuring kadmin [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [1/2]: starting kadmin [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [2/2]: configuring kadmin to start on boot [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Done configuring kadmin. [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Configuring directory server (dirsrv) [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [1/3]: configuring TLS for DS instance [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [2/3]: importing CA certificates from LDAP [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [3/3]: restarting directory server [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Done configuring directory server (dirsrv). [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Configuring the web interface (httpd) [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [1/22]: stopping httpd [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [2/22]: setting mod_nss port to 443 [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [3/22]: setting mod_nss cipher suite [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [4/22]: setting mod_nss protocol list to TLSv1.2 [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [5/22]: setting mod_nss password file [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [6/22]: enabling mod_nss renegotiate [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [7/22]: disabling mod_nss OCSP [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [8/22]: adding URL rewriting rules [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [9/22]: configuring httpd [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [10/22]: setting up httpd keytab [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [11/22]: configuring Gssproxy [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [12/22]: setting up ssl [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [13/22]: configure certmonger for renewals [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [14/22]: importing CA certificates from LDAP [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [15/22]: publish CA cert [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [16/22]: clean up any existing httpd ccaches [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [17/22]: configuring SELinux for httpd [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [18/22]: create KDC proxy config [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [19/22]: enable KDC proxy [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [20/22]: starting httpd [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [21/22]: configuring httpd to start on boot [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [22/22]: enabling oddjobd [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Done configuring the web interface (httpd). [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Configuring ipa-otpd [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [1/2]: starting ipa-otpd [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [2/2]: configuring ipa-otpd to start on boot [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Done configuring ipa-otpd. [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Configuring ipa-custodia [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [1/4]: Generating ipa-custodia config file [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [2/4]: Generating ipa-custodia keys [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [error] SERVER_DOWN: {'desc': "Can't contact LDAP server", 'errno': 107, 'info': 'Transport endpoint is not connected'} [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Your system may be partly configured. [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Run /usr/sbin/ipa-server-install --uninstall to clean up. [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] [ipatests.pytest_ipa.integration.host.Host.replica1.cmd29] Exit code: 1 ipa: ERROR: stderr: ipaserver.install.server.replicainstall: ERROR Could not resolve hostname replica1.ipa.test using DNS. Clients may not function properly. Please check your DNS setup. (Note that this check queries IPA DNS directly and ignores /etc/hosts.) ipapython.admintool: ERROR {'desc': "Can't contact LDAP server", 'errno': 107, 'info': 'Transport endpoint is not connected'} ipapython.admintool: ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information
PR405, logs
The logs end with “OSError: [Errno 113] No route to host” and “paramiko.ssh_exception.SSHException: SSH session not active” in the middle of the installation phase. DS logs on master show “Unable to receive the response for a startReplication extended operation to consumer (Can't contact LDAP server).” Logs were not collected on replica2 due to the paramiko error above. I think this is a infra/network failure and nothing else (but could be wrong).
I support @fcami's assumption, the analysis sounds reasonable. I also ran that several times while testing another issue as a sanity check: https://github.com/freeipa-pr-ci2/freeipa/pull/442.
The test had 10+ green runs, issue happened only once. It was probably an infrastucture issue, closing.
Metadata Update from @frenaud: - Issue close_status updated to: worksforme - Issue status updated to: Closed (was: Open)