From 5a4f24e10e72f922c37bb6bb48cbe4b25b13439b Mon Sep 17 00:00:00 2001 From: William Brown Date: Tue, 3 Oct 2017 13:54:24 +1000 Subject: [PATCH] Ticket 49376 - raise idscanlimit Bug Description: Previously the idlistscanlimit existed to try and coerce searches to operate in a certain pattern, where large indexes like ObjectClass would be avoided, and smaller ones like uid would be preferred, then taking filter_test_thresh and the regex test. However, the issue is that on a query like: '(&(objectClass=x)(uid=y))', the objectClass index would be read up to idlistscanlimit, then discarded everytime, adding needless work to each search. In addition, the cost of the id2entry retrieval and filter test would outweigh the benefit. This is especially apparent on: (|(objectClass=type)(objectClass=type)). So in a database of say 100,000 items, if we did this search with type having 5000 items, this would fold to an expensive ALLIDS search. It is *always* faster to use the index then to access id2entry in this case. In fact, there is no case that I have found where idlistscanlimit is better than proper query optimisation, and even on a suboptimal query idlistscanlimit promotes allids queries that are *always* slower than indexed ones. tl;dr - idlistscanlimit was a construct trying to promote better query arrangement, ie (&(uid=x)(oc=y)), but doing it with an inefficent mechanism. Instead, the use of proper search filter optimisation yields far better results. Fix Description: Set the idlscanlimit to 999,999,999 to prevent it interacting in all cases. It will be removed in a future release. https://pagure.io/389-ds-base/issue/49376 Author: wibrown Review by: ??? --- ldap/servers/slapd/back-ldbm/ldbm_config.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ldap/servers/slapd/back-ldbm/ldbm_config.c b/ldap/servers/slapd/back-ldbm/ldbm_config.c index 2ef4652..d4312be 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_config.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_config.c @@ -1817,7 +1817,7 @@ ldbm_config_db_deadlock_policy_set(void *arg, static config_info ldbm_config[] = { {CONFIG_LOOKTHROUGHLIMIT, CONFIG_TYPE_INT, "5000", &ldbm_config_lookthroughlimit_get, &ldbm_config_lookthroughlimit_set, CONFIG_FLAG_ALWAYS_SHOW | CONFIG_FLAG_ALLOW_RUNNING_CHANGE}, {CONFIG_MODE, CONFIG_TYPE_INT_OCTAL, "0600", &ldbm_config_mode_get, &ldbm_config_mode_set, CONFIG_FLAG_ALWAYS_SHOW | CONFIG_FLAG_ALLOW_RUNNING_CHANGE}, - {CONFIG_IDLISTSCANLIMIT, CONFIG_TYPE_INT, "4000", &ldbm_config_allidsthreshold_get, &ldbm_config_allidsthreshold_set, CONFIG_FLAG_ALWAYS_SHOW | CONFIG_FLAG_ALLOW_RUNNING_CHANGE}, + {CONFIG_IDLISTSCANLIMIT, CONFIG_TYPE_INT, "999999999", &ldbm_config_allidsthreshold_get, &ldbm_config_allidsthreshold_set, CONFIG_FLAG_ALWAYS_SHOW | CONFIG_FLAG_ALLOW_RUNNING_CHANGE}, {CONFIG_DIRECTORY, CONFIG_TYPE_STRING, "", &ldbm_config_directory_get, &ldbm_config_directory_set, CONFIG_FLAG_ALWAYS_SHOW | CONFIG_FLAG_ALLOW_RUNNING_CHANGE | CONFIG_FLAG_SKIP_DEFAULT_SETTING}, {CONFIG_DBCACHESIZE, CONFIG_TYPE_SIZE_T, DEFAULT_CACHE_SIZE_STR, &ldbm_config_dbcachesize_get, &ldbm_config_dbcachesize_set, CONFIG_FLAG_ALWAYS_SHOW | CONFIG_FLAG_ALLOW_RUNNING_CHANGE}, {CONFIG_DBNCACHE, CONFIG_TYPE_INT, "0", &ldbm_config_dbncache_get, &ldbm_config_dbncache_set, CONFIG_FLAG_ALLOW_RUNNING_CHANGE}, -- 1.8.3.1