There seems to be a good number of hosts affected by this, all the below fail on not being able to reach ipa

builders
bodhi-backend
buildvm
mbs front/backend
koji01/02
kojipkgs01/02
odcs
openqa/openqa-workers
osbs
pdc_web
releng_compose
retrace
sign_bridge
qvmhosts

Update:
Missed these 2 off the list
db-koji01
db-openqa01

This is done now, and builders are able to be added.

There's still a number of hosts with fas_client_groups still set. I guess we need to remove that everywhere?

And running -t ipa/client over everything:

• buildvm-s390x-01.stg.s390.fedoraproject.org fails with: Skip ipa01.stg.iad2.fedoraproject.org: LDAP server is not responding, unable to verify if this is an IPA server

• buildvm-s390x-* fails with:

Skip ipa02.iad2.fedoraproject.org: LDAP server is not responding, unable to verify if this is an IPA server\nSkip ipa03.iad2.fedoraproject.org: LDAP server is not responding, unable to verify if this is an IPA server\nSkip ipa01.iad2.fedoraproject.org: 
LDAP server is not responding, unable to verify if this is an IPA server\nSkip ipa02.iad2.fedoraproject.or
g: LDAP server is not responding, unable to verify if this is an IPA server\nSkip ipa01.iad2.fedoraproject
.org: LDAP server is not responding, unable to verify if this is an IPA server\nSkip ipa03.iad2.fedoraproj
ect.org: LDAP server is not responding, unable to verify if this is an IPA server\nUnable to find IPA Serv
er to join\nThe ipa-client-install command failed. See /var/log/ipaclient-install.log for more information
", "stderr_lines": ["Skip ipa02.iad2.fedoraproject.org: LDAP server is not responding, unable to verify if
 this is an IPA server", "Skip ipa03.iad2.fedoraproject.org: LDAP server is not responding, unable to veri
fy if this is an IPA server", "Skip ipa01.iad2.fedoraproject.org: LDAP server is not responding, unable to
 verify if this is an IPA server", "Skip ipa02.iad2.fedoraproject.org: LDAP server is not responding, unab
le to verify if this is an IPA server", "Skip ipa01.iad2.fedoraproject.org: LDAP server is not responding,
 unable to verify if this is an IPA server", "Skip ipa03.iad2.fedoraproject.org: LDAP server is not respon
ding, unable to verify if this is an IPA server", "Unable to find IPA Server to join", "The ipa-client-ins
tall command failed. See /var/log/ipaclient-install.log for more information"

Here's the failures:

batcave13.rdu2.fedoraproject.org
buildvm-ppc64le-.stg.iad2.fedoraproject.org
ns13.rdu2.fedoraproject.org
buildvm-s390x-

These should be vpn I think? Or not have any ipa/client
retrace-stg.aws.fedoraproject.org
retrace03.rdu-cc.fedoraproject.org

On Mon, 29 Mar 2021 at 22:40, Kevin Fenzi pagure@pagure.io wrote:

kevin added a new comment to an issue you are following:
``
This is done now, and builders are able to be added.

There's still a number of hosts with fas_client_groups still set. I guess
we need to remove that everywhere?

And running -t ipa/client over everything:

• buildvm-s390x-01.stg.s390.fedoraproject.org fails with: Skip
  ipa01.stg.iad2.fedoraproject.org: LDAP server is not responding, unable
  to verify if this is an IPA server

• buildvm-s390x-* fails with:

Skip ipa02.iad2.fedoraproject.org: LDAP server is not responding, unable to verify if this is an IPA server\nSkip ipa03.iad2.fedoraproject.org: LDAP server is not responding, unable to verify if this is an IPA server\nSkip ipa01.iad2.fedoraproject.org: LDAP server is not responding, unable to verify if this is an IPA server\nSkip ipa02.iad2.fedoraproject.or g: LDAP server is not responding, unable to verify if this is an IPA server\nSkip ipa01.iad2.fedoraproject .org: LDAP server is not responding, unable to verify if this is an IPA server\nSkip ipa03.iad2.fedoraproj ect.org: LDAP server is not responding, unable to verify if this is an IPA server\nUnable to find IPA Serv er to join\nThe ipa-client-install command failed. See /var/log/ipaclient-install.log for more information ", "stderr_lines": ["Skip ipa02.iad2.fedoraproject.org: LDAP server is not responding, unable to verify if this is an IPA server", "Skip ipa03.iad2.fedoraproject.org: LDAP server is not responding, unable to veri fy if this is an IPA server", "Skip ipa01.iad2.fedoraproject.org: LDAP server is not responding, unable to verify if this is an IPA server", "Skip ipa02.iad2.fedoraproject.org: LDAP server is not responding, unab le to verify if this is an IPA server", "Skip ipa01.iad2.fedoraproject.org: LDAP server is not responding, unable to verify if this is an IPA server", "Skip ipa03.iad2.fedoraproject.org: LDAP server is not respon ding, unable to verify if this is an IPA server", "Unable to find IPA Server to join", "The ipa-client-ins tall command failed. See /var/log/ipaclient-install.log for more information"

Here's the failures:

batcave13.rdu2.fedoraproject.org
buildvm-ppc64le-.stg.iad2.fedoraproject.org
ns13.rdu2.fedoraproject.org
buildvm-s390x-

These should be vpn I think? Or not have any ipa/client
retrace-stg.aws.fedoraproject.org
retrace03.rdu-cc.fedoraproject.org

we didn't want them to be on vpn because they run untrusted core dumps.
This goes with the copr boxes. This worked before with FAS because it was a
push mechanism. We will have to come up with a different method for these
sorts of hosts.

``

To reply, visit the link below or just reply to this email
https://pagure.io/fedora-infrastructure/issue/9769

--
Stephen J Smoogen.

So, we are down to:

• s390x stuff (both prod and stg). I think @smooge was going to submit another ticket on this? Or should we ask someone else to based on the howto now?

• retrace-stg (might be in the wrong group?)

• bvmhost-p09-01 (has some kind of issue, will retry after it's rebooted)

There's another ticket in for s390x items.

retrace-stg and bvmhost-p09-01 should be dealt with.

The s390x builders are now all enrolled in ipa. :)

Feel free to re-open if you see any machines we missed, but I think we got them all.