#17 slapi-nis can add empty sdn in base search list of attributes used to update references
Opened by tbordaz. Modified

ipa-server-4.4.0-14.el7_3.7.x86_64
slapi-nis-0.56.0-4.el7.x86_64

During installation of a freeipa replica, slapi-nis can store empty sdn in a map structure used to update references. The sdn are used as base search for internal searches.

Because of bug https://pagure.io/389-ds-base/issue/49291, it triggers that DS crash. (see backstack below)

This bug is also related to https://pagure.io/slapi-nis/issue/16

The bug is that slapi-nis should ignore empty sdn.

    Thread 1 (Thread 0x7fa53bfe7700 (LWP 18900)):
    #0  0x00007fa579a22711 in send_ldap_result_ext (pb=pb@entry=0x7fa528007550, err=err@entry=53, matched=matched@entry=0x0, 
        text=text@entry=0x7fa579a4c708 "This plugin is not configured to access operation target data", 
        nentries=nentries@entry=0, 
        urls=urls@entry=0x0, ber=ber@entry=0x0) at ldap/servers/slapd/result.c:353
    #1  0x00007fa579a22fd1 in send_ldap_result (pb=pb@entry=0x7fa528007550, err=err@entry=53, matched=matched@entry=0x0, 
        text=text@entry=0x7fa579a4c708 "This plugin is not configured to access operation target data", 
        nentries=nentries@entry=0, 
        urls=urls@entry=0x0) at ldap/servers/slapd/result.c:194
    #2  0x00007fa579a13260 in slapi_search_internal_callback_pb (pb=pb@entry=0x7fa528007550, 
        callback_data=callback_data@entry=0x7fa53bfe5ea0, prc=prc@entry=0x0, 
        psec=psec@entry=0x7fa56a802df0 <backend_shr_note_entry_sdn_cb>, prec=prec@entry=0x0)
        at ldap/servers/slapd/plugin_internal_op.c:559
    #3  0x00007fa56a804a61 in backend_shr_update_references_cb (group=0x7fa520007d50 "cn=compat,<suffix>", 
        set=0x7fa52000aa50 "cn=groups", flag=<optimized out>, backend_data=0x7fa52392daa0, cbdata_ptr=<optimized out>)
        at back-shr.c:1539
    #4  0x00007fa56a81203f in map_data_foreach_map (state=state@entry=0x7fa57b585a30, domain_name=domain_name@entry=0x0, 
        fn=fn@entry=0x7fa56a803fd0 <backend_shr_update_references_cb>, cbdata=cbdata@entry=0x7fa53bfe5f50) at map.c:347
    #5  0x00007fa56a80279b in backend_shr_update_references (state=0x7fa57b585a30, pb=pb@entry=0x7fa53bfe6a90, 
        e=<optimized out>, 
        mods=mods@entry=0x0, modlist=modlist@entry=0x0) at back-shr.c:1825
    #6  0x00007fa56a803a17 in backend_shr_add_cb (pb=pb@entry=0x7fa53bfe6a90) at back-shr.c:1953
    #7  0x00007fa56a803b41 in backend_shr_add_cb (pb=0x7fa53bfe6a90) at back-shr.c:1883
    #8  backend_shr_betxn_post_add_cb (pb=0x7fa53bfe6a90) at back-shr.c:1965
    #9  0x00007fa579a0db18 in plugin_call_func (list=0x7fa57b58e910, operation=operation@entry=560, 
        pb=pb@entry=0x7fa53bfe6a90, 
        call_one=call_one@entry=0) at ldap/servers/slapd/plugin.c:2049
    #10 0x00007fa579a0dda3 in plugin_call_list (pb=0x7fa53bfe6a90, operation=560, list=<optimized out>)
        at ldap/servers/slapd/plugin.c:1993
    #11 plugin_call_plugins (pb=pb@entry=0x7fa53bfe6a90, whichfunction=whichfunction@entry=560) at 
        ldap/servers/slapd/plugin.c:445
    #12 0x00007fa56c7b9b94 in ldbm_back_add (pb=0x7fa53bfe6a90) at ldap/servers/slapd/back-ldbm/ldbm_add.c:1150
    #13 0x00007fa5799b1ed8 in op_shared_add (pb=pb@entry=0x7fa53bfe6a90) at ldap/servers/slapd/add.c:699
    #14 0x00007fa5799b31a0 in do_add (pb=pb@entry=0x7fa53bfe6a90) at ldap/servers/slapd/add.c:226
    #15 0x00007fa579ee59c3 in connection_dispatch_operation (pb=0x7fa53bfe6a90, op=0x7fa57b7521b0, conn=0x7fa57bc05118)
        at ldap/servers/slapd/connection.c:612
    #16 connection_threadmain () at ldap/servers/slapd/connection.c:1759
    #17 0x00007fa577bdb9bb in _pt_root (arg=0x7fa57bbcf3d0) at ../../../nspr/pr/src/pthreads/ptthread.c:216
    #18 0x00007fa57757bdc5 in start_thread (arg=0x7fa53bfe7700) at pthread_create.c:308
    #19 0x00007fa5772aa76d in __lseek_nocancel () at ../sysdeps/unix/syscall-template.S:81
    #20 0x0000000000000000 in ?? ()

  • The crash occurred while only one thread was active

  • The reason of the crash is https://pagure.io/slapi-nis/issue/16
    But the conditions that lead to that crash are the purpose of this ticket

  • slapi-nis maps two domains

    # 2 domains: 
    #   compat
    #   sudoers
    #
    (gdb) print map_data
    $1 = {master = 0x0, domains = 0x7fa523d3bb80, n_domains = 2, lock = 0x7fa57b58c2f0}
    (gdb) print map_data.domains[0]
    $2 = {name = 0x7fa520007d50 "cn=compat,<suffix>", maps = 0x7fa51339ce20, n_maps = 4}
    (gdb) print map_data.domains[1]
    $4 = {name = 0x7fa510f039b0 "ou=sudoers,<suffix>", maps = 0x7fa520006a30, n_maps = 1}
    
  • Domain 'compat' has 4 maps

    # Domain: compat
    #   map: cn=users
    #   map: cn=ng
    #   map: cn=groups
    #   map: cn=computers
    #
    (gdb) print ((struct map) map_data.domains[0]->maps[0])
    $9 = {name = 0x7fa520003850 "cn=users", last_changed = 1497378023, secure = 0, entries = 0x7fa51075fda0, 
      n_unique_entries = 10448, 
      n_key_trees = 1, key_trees = 0x7fa520263f00, id_tree = 0x7fa520265a60, backend_data = 0x7fa520001b60, 
      free_backend_data = 0x7fa56a8024c0 <backend_shr_set_config_free_config>}
    (gdb) print ((struct map) map_data.domains[0]->maps[1])
    $10 = {name = 0x7fa520008420 "cn=ng", last_changed = 1497378024, secure = 0, entries = 0x7fa5100828c0, 
      n_unique_entries = 38, 
      n_key_trees = 1, key_trees = 0x7fa523e09020, id_tree = 0x7fa510bd20b0, backend_data = 0x7fa510b94070, 
      free_backend_data = 0x7fa56a8024c0 <backend_shr_set_config_free_config>}
    (gdb) print ((struct map) map_data.domains[0]->maps[2])
    $11 = {name = 0x7fa52000aa50 "cn=groups", last_changed = 1497378037, secure = 0, entries = 0x7fa511decbc0, 
      n_unique_entries = 6823, n_key_trees = 1, key_trees = 0x7fa510f8b130, id_tree = 0x7fa5120fa090, 
      backend_data = 0x7fa52392daa0, 
      free_backend_data = 0x7fa56a8024c0 <backend_shr_set_config_free_config>}
    (gdb) print ((struct map) map_data.domains[0]->maps[3])
    $12 = {name = 0x7fa5104585c0 "cn=computers", last_changed = 1497378037, secure = 0, entries = 0x0, 
      n_unique_entries = 0, 
      n_key_trees = 0, key_trees = 0x0, id_tree = 0x0, backend_data = 0x7fa5122c4730, 
      free_backend_data = 0x7fa56a8024c0 <backend_shr_set_config_free_config>}
    
  • The crash occurred while evaluating references of 'member' attribute in 'cn=groups' map

    (gdb) print *((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)
    $34 = {state = 0x7fa57b585a30, group = 0x7fa521a75df0 "cn=compat,<suffix>", set = 0x7fa5238e28a0 "cn=groups", 
      bases = 0x7fa5238ea0c0, entry_filter = 0x7fa520008220 "(objectclass=posixGroup)", rel_attrs = 0x7fa523eae650, 
      rel_attr_list = 0x7fa528007860 "cn,ipaanchoruuid,gidNumber,memberUid,ipauniqueid,member,uid", 
      rel_attrs_list = 0x7fa523eae650, 
      ref_attrs = 0x0, inref_attrs = 0x0, ref_attr_list = 0x7fa510582480, inref_attr_list = 0x0, 
      skip_uninteresting_updates = 1, 
      restrict_subtrees = 0x0, ignore_subtrees = 0x7fa523917fd0, self = 0x7fa52392daa0}
    # only one attribute in that map
    (gdb) print *((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)->ref_attr_list[0]
    $35 = {group = 0x7fa523eae720 "cn=compat,<suffix>", set = 0x7fa523eae750 "cn=groups", 
      links = 0x7fa510458fe0, n_links = 2}
    (gdb) print *((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)->ref_attr_list[1]
    Cannot access memory at address 0x0
    # attribute is 'member'
    (gdb) print *((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)->ref_attr_list[0]->links
    $36 = {attribute = 0x7fa523eae6e0 "member", filter_str = 0x0, filter = 0x0, base_sdn_list = 0x7fa512b598e0, 
      base_sdn_list2 = 0x7fa512b59910}
    
  • The crash occurred because the base sdn list, that should be NULL terminated, contains an empty base

    # base suffixes
    #   "cn=groups,cn=accounts,<suffix>"
    (gdb) print *((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)->
    ref_attr_list[0]->links->base_sdn_list[0]
    $39 = {flag = 15 '\017', udn = 0x7fa510f8b210 "cn=groups,cn=accounts,<suffix>", 
      dn = 0x7fa523e64b60 "cn=groups,cn=accounts,<suffix>", 
      ndn = 0x7fa523e64a20 "cn=groups,cn=accounts,<suffix>", 
      ndn_len = 37}
    #   "cn=users,cn=accounts,<suffix>"
    (gdb) print *((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)->
      ref_attr_list[0]->links->base_sdn_list[1]
    $40 = {flag = 15 '\017', udn = 0x7fa523e64b30 "cn=users,cn=accounts,<suffix>", 
      dn = 0x7fa5102f3200 "cn=users,cn=accounts,<suffix>", 
      ndn = 0x7fa5102f30e0 "cn=users,cn=accounts,<suffix>", 
      ndn_len = 36}
    #  empty sdn
    (gdb) print *((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)->
      ref_attr_list[0]->links->base_sdn_list[2]
    $41 = {flag = 9 '\t', udn = 0x0, dn = 0x0, ndn = 0x0, ndn_len = 0}
    
  • The base sdn has been added with format_add_sdn_list. This fonction adds the sdn in both lists
    base_sdn_list (unsorted) and base_sdn_list2 (sorted)

    #
    # unsorted list
    gdb) print ((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)->
    ref_attr_list[0]->links[0]->base_sdn_list[0]
    $1 = (struct slapi_dn *) 0x7fa510f8b1e0
    (gdb) print ((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)->
    ref_attr_list[0]->links[0]->base_sdn_list[1]
    $2 = (struct slapi_dn *) 0x7fa523e64c50
    (gdb) print ((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)->
    ref_attr_list[0]->links[0]->base_sdn_list[2]
    $3 = (struct slapi_dn *) 0x7fa512b59a70
    #
    # unsorted list contains the same sdn address
    (gdb) print ((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)->
    ref_attr_list[0]->links[0]->base_sdn_list2[0]
    $4 = (struct slapi_dn *) 0x7fa512b59a70
    (gdb) print ((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)->
    ref_attr_list[0]->links[0]->base_sdn_list2[1]
    $5 = (struct slapi_dn *) 0x7fa510f8b1e0
    (gdb) print ((struct backend_shr_set_data *)((struct map) map_data.domains[0]->maps[2])->backend_data)->
    ref_attr_list[0]->links[0]->base_sdn_list2[2]
    $6 = (struct slapi_dn *) 0x7fa523e64c50
    

So there was no corruption of the lists

  • Looking at all format_add_sdn_list calls, nothing prevents adding an empty sdn.
    No obvious bug found while investigating those calls.

  • Nothing in error logs can reveal when empty sdn was added

In conclusion:
The crash occurred because at some time format_add_sdn_list was called to add an empty sdn
to the 'member' attribute of the 'cn=groups' map.
No tips to know when/why it happened

Next steps

  • We can inverstigate more the conditions at each calls of format_add_sdn_list
    but it would be very time consuming without guarantie to find the RC

  • I suggest to harden format_add_sdn_list, so that it does not add empty sdn

A reproducible test case should be to create a group entry, that slapi-nis will map, with a corrupted member attribute that contains an empty 'member' value.
Details are attached in the case

Metadata