freeipa

#7790 ipa host-del --updatedns FQDN yeilds unindexed searches
Opened by jaredl. Modified

Open
Close Issue

Issue

Currently, when we execute the command ipa host-del --updatedns $fqdn, there's 7 unindexed searches performed. Most of these are returning in < 0.0## seconds per their etime however, one of them has an etime of ~11 seconds.

Additionally, the host-del operation is performing a substring match, which isn't ideal and is expensive.

Steps to Reproduce

  1. Install FreeIPA v4.5.4 on CentOS 7
  2. Add some hosts
  3. For the hosts, add some extra services via ipa service-add
  4. On the hosts, get a certificate for the services.
  5. Decommission the host, and run ipa host-del --updatedns $fqdn
  6. From the dirsrv access logs, notice that there's a few unindexed searches.

Actual behavior

The host deletion method runs a few unindexed searches, of which, one is specifically terrible at takes ~11 seconds.

Expected behavior

Most/all of the queries done by the host-del operations are indexed properly out of the box.

Additionally, if possible, instead of performing the substring match during the host-del, we should look into if it's possible to perform an equality match instead since maintaining the substring index is expensive.

Version/Release/Distribution

rpm -q freeipa-server freeipa-client ipa-server ipa-client 389-ds-base pki-ca krb5-server
package freeipa-server is not installed
package freeipa-client is not installed
ipa-server-4.5.4-10.el7.centos.4.4.x86_64
ipa-client-4.5.4-10.el7.centos.4.4.x86_64
389-ds-base-1.3.7.5-28.el7_5.x86_64
pki-ca-10.5.1-15.el7_5.noarch
krb5-server-1.15.1-19.el7.x86_64

Additional info:

I've used logconv.pl on the access logs from the host-del operations, the unindexed pieces are:

  Unindexed Component #1 (notes=U)
  -  Date/Time:             03/Dec/2018:17:42:08
  -  Connection Number:     289891
  -  Operation Number:      16
  -  Etime:                 0.0000473303
  -  Nentries:              1
  -  IP Address:            172.16.1.81
  -  Search Base:           cn=ops-ovc-ipa-1.ops.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
  -  Search Scope:          2 (subtree)
  -  Search Filter:         (&(objectclass=ipaconfigobject)(ipaconfigstring=enabledservice)(cn=ca))
  -  Bind DN:               uid=admin,cn=users,cn=accounts,dc=example,dc=com
  Unindexed Component #2 (notes=U)
  -  Date/Time:             03/Dec/2018:17:42:20
  -  Connection Number:     289891
  -  Operation Number:      25
  -  Etime:                 0.0000471310
  -  Nentries:              1
  -  IP Address:            172.16.1.81
  -  Search Base:           cn=ops-ovc-ipa-1.ops.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
  -  Search Scope:          2 (subtree)
  -  Search Filter:         (&(objectclass=ipaconfigobject)(ipaconfigstring=enabledservice)(cn=ca))
  -  Bind DN:               uid=admin,cn=users,cn=accounts,dc=example,dc=com
  Unindexed Component #3 (notes=U)
  -  Date/Time:             03/Dec/2018:17:42:31
  -  Connection Number:     289891
  -  Operation Number:      45
  -  Etime:                 0.0000490972
  -  Nentries:              1
  -  IP Address:            172.16.1.81
  -  Search Base:           cn=ops-ovc-ipa-1.ops.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
  -  Search Scope:          2 (subtree)
  -  Search Filter:         (&(objectclass=ipaconfigobject)(ipaconfigstring=enabledservice)(cn=ca))
  -  Bind DN:               uid=admin,cn=users,cn=accounts,dc=example,dc=com
  Unindexed Component #4 (notes=U)
  -  Date/Time:             03/Dec/2018:17:42:43
  -  Connection Number:     289891
  -  Operation Number:      65
  -  Etime:                 0.0000502800
  -  Nentries:              1
  -  IP Address:            172.16.1.81
  -  Search Base:           cn=ops-ovc-ipa-1.ops.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
  -  Search Scope:          2 (subtree)
  -  Search Filter:         (&(objectclass=ipaconfigobject)(ipaconfigstring=enabledservice)(cn=ca))
  -  Bind DN:               uid=admin,cn=users,cn=accounts,dc=example,dc=com
  Unindexed Component #5 (notes=U)
  -  Date/Time:             03/Dec/2018:17:42:54
  -  Connection Number:     289891
  -  Operation Number:      85
  -  Etime:                 0.0000454166
  -  Nentries:              1
  -  IP Address:            172.16.1.81
  -  Search Base:           cn=ops-ovc-ipa-1.ops.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
  -  Search Scope:          2 (subtree)
  -  Search Filter:         (&(objectclass=ipaconfigobject)(ipaconfigstring=enabledservice)(cn=ca))
  -  Bind DN:               uid=admin,cn=users,cn=accounts,dc=example,dc=com
  Unindexed Component #6 (notes=U)
  -  Date/Time:             03/Dec/2018:17:43:06
  -  Connection Number:     289891
  -  Operation Number:      105
  -  Etime:                 0.0000461436
  -  Nentries:              1
  -  IP Address:            172.16.1.81
  -  Search Base:           cn=ops-ovc-ipa-1.ops.example.com,cn=masters,cn=ipa,cn=etc,dc=example,dc=com
  -  Search Scope:          2 (subtree)
  -  Search Filter:         (&(objectclass=ipaconfigobject)(ipaconfigstring=enabledservice)(cn=ca))
  -  Bind DN:               uid=admin,cn=users,cn=accounts,dc=example,dc=com
  Unindexed Component #7 (notes=U)
  -  Date/Time:             03/Dec/2018:17:42:08
  -  Connection Number:     289891
  -  Operation Number:      4
  -  Etime:                 11.0094442803
  -  Nentries:              5
  -  IP Address:            172.16.1.81
  -  Search Base:           cn=services,cn=accounts,dc=example,dc=com
  -  Search Scope:          1 (one)
  -  Search Filter:         (&(&(objectclass=ipaservice)(!(objectclass=posixaccount))(!(|(krbprincipalname=kadmin/*)(krbprincipalname=k/m@*)(krbprincipalname=krbtgt/*))))(&(|(managedby=*blah-abc-hoopla-1.ops.example.com*)(krbprincipalname=*blah-abc-hoopla-1.ops.example.com*)(ipakrbauthzdata=*blah-abc-hoopla-1.ops.example.com*))(&(objectclass=krbprincipal)(objectclass=krbprincipalaux)(objectclass=krbticketpolicyaux)(objectclass=ipaobject)(objectclass=ipaservice)(objectclass=pkiuser))))
  -  Bind DN:               uid=admin,cn=users,cn=accounts,dc=example,dc=com

Talking on IRC with @mreynolds and @cheimes , it looks like we should have an eq and sub index for ipakrbauthzdata. For what it's worth, we currently have the following indexes:

# ldapsearch -D "cn=directory manager" -W -b "cn=index,cn=userroot,cn=ldbm database,cn=plugins,cn=config" -xLLL '(|(cn=managedby)(cn=krbprincipalname)(cn=ipakrbauthzdata))'
Enter LDAP Password:
dn: cn=krbPrincipalName,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=co
 nfig
cn: krbPrincipalName
nsIndexType: eq
nsIndexType: sub
nsMatchingRule: caseIgnoreIA5Match
nsMatchingRule: caseExactIA5Match
nsSystemIndex: false
objectClass: top
objectClass: nsIndex
dn: cn=managedby,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
cn: managedby
nsIndexType: eq
nsIndexType: pres
nsIndexType: sub
nsSystemIndex: false
objectClass: top
objectClass: nsIndex

Summary

Missing index on ipakrbauthzdata

The query is slow because the ipakrbauthzdata is not index. The term is used in the OR filter managedby, krbprincipalname, and ipakrbauthzdata.

dn: cn=ipakrbauthzdata,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
objectClass: top
objectClass: nsIndex
cn: ipakrbauthzdata
nsIndexType: eq
nsIndexType: pres
nsIndexType: sub
nsSystemIndex: false

full substring match

Further more the search is slow because it involves a lot of substring matches. I think there is room to reduce the amount of wildcards in the query:

  • (managedby=*blah-abc-hoopla-1.ops.example.com*) could be replaced by (managedby=fqdn=blah-abc-hoopla-1.ops.example.com,cn=computers,cn=accounts,$SUFFIX)
  • (krbprincipalname=*blah-abc-hoopla-1.ops.example.com*) could be replaced by (krbprincipalname=*/blah-abc-hoopla-1.ops.example.com@$REALM)
  • I don't yet know if (ipakrbauthzdata=*blah-abc-hoopla-1.ops.example.com*) can be substituted by another query with less or no wildcards.

The proposed queries look safer to me, too. For example the improved krbprincipalname query would no longer match a principal of host blahblah-abc-hoopla-1.ops.example.com

Metadata Update from @cheimes:
- Issue set to the milestone: FreeIPA 4.6.5

Metadata Update from @cheimes:
- Issue tagged with: performance

Hey @cheimes - In the above you have nsIndexType: pres however, if I understand correct from our conversation on IRC yesterday, ipakrbauthzdata only needs an eq and sub index. Is that correct?

Metadata Update from @cheimes:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/2649

master:

  • 0fb87bfe827b10af25fcd35c327b33977053e0e3 LDAPUpdate: Batch index tasks
  • ed436e4b62c9d3ec02111d486ba77bd09e9838fc Add more LDAP indices
  • a34d92d25c97feb778d4c222704d8ac32e146768 Create reindex task for ipaca DB

ipa-4-6:

  • 55f18a8bd51267dacf6d048ac94179c1a44714b0 LDAPUpdate: Batch index tasks
  • f29ab7730b3b8866f9a10ddaffb0b828b53b1436 Add more LDAP indices
  • 26d90cdd69641d07a11b8f8ddbd340aded42280d Create reindex task for ipaca DB

ipa-4-7:

  • ad37e0c2b76afda289b7795f518ae9a1e0703151 LDAPUpdate: Batch index tasks
  • 903cfe390311b8e21273ceeb2a889974aabc1708 Add more LDAP indices
  • e484c4ba0a54813b84fffab5944d5ccec7a2c3b8 Create reindex task for ipaca DB

The new indices will be available in 4.7.3 and next 4.6 update.

I'm leaving this bug open to investigate if some of the queries and indices can be replaced by non-wildcard matches.
Metadata
None

performance

None
None
None
None
https://github.com/freeipa/freeipa/pull/2649
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.