#9777 kdb: ipadb_get_connection() succeeds but returns null LDAP context
Closed: fixed by sumenon. Opened by jrische.

In environments where 389ds is overwhelmed, KDC crashes are happening as part of ipadb_simple_search(), more precisely when calling the openldap API, which is surprising because the connection with 389ds is supposed to be checked before that point:

krb5_error_code ipadb_simple_search(struct ipadb_context *ipactx,
                                    char *basedn, int scope,
                                    char *filter, char **attrs,
                                    LDAPMessage **res)
{
    int ret;
    ret = ipadb_check_connection(ipactx);
    if (ret != 0)
        return ipadb_simple_ldap_to_kerr(ret);
    ret = ldap_search_ext_s(ipactx->lcontext, basedn, scope, /* KDC crash here */
                            filter, attrs, 0, NULL, NULL,
                            &std_timeout, LDAP_NO_LIMIT,
                            res);

The connection check function tries to re-establish the connection if the LDAP context (ipactx->lcontext) is NULL. Error handling seems to be done properly in all the functions mentioned so far, which pushes me to think there is likely a problem with the ipadb_get_connection() function.

KDC crashes occur just after a failed attempt to configure the PAC generator.

krb5kdc[xxxxx]: MS-PAC generator: Failed to fetch trusted domains information
systemd[1]: Started Process Core Dump (PID xxxxx/UID 0).

This kind of messages are returned by the ipadb_reinit_mspac() function, which may call ipadb_mspac_check_trusted_domains() where ipadb_simple_search() is called too, which might result in a ipadb_get_connection() call, if a connection failure happens at this point, the LDAP context will be reset to NULL:

ipadb_reinit_mspac():

            err = ipadb_mspac_check_trusted_domains(ipactx);
            if (err) {
                if (err == KRB5_KDB_NOENTRY) {
                    /* SKIP */
                    err = 0;
                } else {
                    in_stmsg = "Failed to fetch trusted domains information"; /* message just before KDC crash */
                }
                goto end;
            }
{code}
``ipadb_mspac_check_trusted_domains()``:
```c
    ret = ipadb_simple_search(ipactx, base, LDAP_SCOPE_SUBTREE,
                              filter, attrs, &result);
done:
    ldap_msgfree(result);
    free(base);
    return ret;

The problem is that the final step in ipadb_get_connection() is also calling ipadb_reinit_mspac(). There are clearly recursion and error handling problems here:

    ret = ipadb_reinit_mspac(ipactx, false, &stmsg); /* connection lost, ipactx->lcontext is NULL */
    if (ret && stmsg)
        krb5_klog_syslog(LOG_WARNING, "MS-PAC generator: %s", stmsg);
    ret = 0; /* connection error lost here */
done:
    ldap_msgfree(res);
    if (ret) { /* ret == 0, skipped */
        if (ipactx->lcontext) {
            ldap_unbind_ext_s(ipactx->lcontext, NULL, NULL);
            ipactx->lcontext = NULL;
        }
        if (ret == LDAP_SERVER_DOWN) {
            return ETIMEDOUT;
        }
        return EIO;
    }
    return 0; /* no error returned, ipactx->lcontext still NULL */

Non-zero return codes from ipadb_reinit_mspac() are not treated as fatal error because the absence of the LDAP entries needed to configure the PAC generator are not strictly required for the KDC to function. However, a connection lost error in this context is fatal because it will be ignored, and the final part of the function returning an error code if the LDAP context is NULL will not be executed.

Since no error was raised, ipadb_simple_search() will call ldap_search_ext_s() with a NULL LDAP context.


Metadata Update from @jrische:
- Custom field rhbz adjusted to https://issues.redhat.com/browse/RHEL-88833 https://issues.redhat.com/browse/RHEL-89145 https://issues.redhat.com/browse/RHEL-88834 https://issues.redhat.com/browse/RHEL-89144 https://issues.redhat.com/browse/RHEL-89143 https://issues.redhat.com/browse/RHEL-89142 https://issues.redhat.com/browse/RHEL-89141 https://issues.redhat.com/browse/RHEL-89140 https://issues.redhat.com/browse/RHEL-58453 https://issues.redhat.com/browse/RHEL-89149 https://issues.redhat.com/browse/RHEL-89148 https://issues.redhat.com/browse/RHEL-89147 https://issues.redhat.com/browse/RHEL-89146

Metadata Update from @jrische:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/7790

Metadata Update from @jrische:
- Custom field rhbz adjusted to https://issues.redhat.com/browse/RHEL-88833 https://issues.redhat.com/browse/RHEL-89145 https://issues.redhat.com/browse/RHEL-88834 https://issues.redhat.com/browse/RHEL-89144 https://issues.redhat.com/browse/RHEL-89143 https://issues.redhat.com/browse/RHEL-89142 https://issues.redhat.com/browse/RHEL-89141 https://issues.redhat.com/browse/RHEL-89140 https://issues.redhat.com/browse/RHEL-58453 https://issues.redhat.com/browse/RHEL-89149 https://issues.redhat.com/browse/RHEL-89148 (was: https://issues.redhat.com/browse/RHEL-88833 https://issues.redhat.com/browse/RHEL-89145 https://issues.redhat.com/browse/RHEL-88834 https://issues.redhat.com/browse/RHEL-89144 https://issues.redhat.com/browse/RHEL-89143 https://issues.redhat.com/browse/RHEL-89142 https://issues.redhat.com/browse/RHEL-89141 https://issues.redhat.com/browse/RHEL-89140 https://issues.redhat.com/browse/RHEL-58453 https://issues.redhat.com/browse/RHEL-89149 https://issues.redhat.com/browse/RHEL-89148 https://issues.redhat.com/browse/RHEL-89147 https://issues.redhat.com/browse/RHEL-89146)

master:

  • 56261bbba4355c33a002df98566b290ef9681c0c kdb: keep ipadb_get_connection() from succeeding with null LDAP context

ipa-4-9:

  • ae37b3e6ed12bddb650bdce8e9729e81fef40840 kdb: keep ipadb_get_connection() from succeeding with null LDAP context

ipa-4-10:

  • 088fc98c13c7525410c0667bf6aae388b156c82f kdb: keep ipadb_get_connection() from succeeding with null LDAP context

ipa-4-11:

  • 52c45de2b5272c3613fd87148c1a8f4d361b3eb3 kdb: keep ipadb_get_connection() from succeeding with null LDAP context

ipa-4-12:

  • 5d893c9c3b8d384873f40d2524b1ebf0f34fb452 kdb: keep ipadb_get_connection() from succeeding with null LDAP context

Closing the ticket.

Metadata Update from @sumenon:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata