It appears these BDB files in /var/tmp are completely controlled by the bdb library. I tried to see if I could flush them by periodically "untie"ing the files and reopening them. While this does clear out those files, as soon as we "tie" the files again and make updates, those BDB files reappear. Their size remains the same as it was when they were untied, and then they continue to grow as new updates come in.

Are the BDB files in /var/tmp created for the tied hashes, tied arrays, or both?
Is there any way to make them use something other than /var/tmp?
We may have to skip using tied hashes/arrays and just use the DB_File bdb interface directly . . .

Replying to [comment:5 rmeggins]:

Are the BDB files in /var/tmp created for the tied hashes, tied arrays, or both?

Actually I have only seen 12 files max in my testing. Your test machine doesn't appear to be around anymore. So initially it looks like this is only for the hashes(16 hashes possible)? Do you recall how many you saw in your testing? Also, I do not know if it's the hashes, or simply once the DB file gets past a certain size that these other files are created.

Is there any way to make them use something other than /var/tmp?

Not that I could find; not using the perl interface at least.

We may have to skip using tied hashes/arrays and just use the DB_File bdb interface directly . . .

Maybe, but it's possible it might still create these files.

reassigning to Rich since he is working on the fix.

logconv wip - logconv.pl -V is about 30 times faster
logconv.pl

ugh - need to clean up formatting

0001-Ticket-47501-logconv.pl-uses-var-tmp-for-BDB-temp-fi.patch
0001-Ticket-47501-logconv.pl-uses-var-tmp-for-BDB-temp-fi.patch

same patch but ignore whitespace diffs
ignore-ws.patch

Looks good, except can you bump the version number at the top of the script please? Otherwise, ack.

30 times faster! It's amazing how much the array/DB_RECNO had an impact on performance.

To ssh://git.fedorahosted.org/git/389/ds.git
e6eab21..4d20922 master -> master
commit 4d20922423c9e22c858e0be7dd317591631cf357
Author: Rich Megginson rmeggins@redhat.com
Date: Tue Jul 23 17:05:32 2013 -0600

Ticket has been cloned to Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1013163

To ssh://git.fedorahosted.org/git/389/ds.git
fbece32..313dd8e 389-ds-base-1.2.11 -> 389-ds-base-1.2.11
commit e53b8ea1db80241c1b15541bfed05dbc8fdabb25
Author: Rich Megginson rmeggins@redhat.com
Date: Tue Jul 23 17:05:32 2013 -0600

To ssh://git.fedorahosted.org/git/389/ds.git
c96eaa0..9103b3e 389-ds-base-1.3.0 -> 389-ds-base-1.3.0
commit 223c0535e09b46b3024dc9977eee048f824f2b24
Author: Rich Megginson rmeggins@redhat.com
Date: Tue Jul 23 17:05:32 2013 -0600

To ssh://git.fedorahosted.org/git/389/ds.git
ff9a292..e38685b 389-ds-base-1.3.1 -> 389-ds-base-1.3.1
commit d890e65b4ab15d0a836dbb248da8606867938630
Author: Rich Megginson rmeggins@redhat.com
Date: Tue Jul 23 17:05:32 2013 -0600

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/838

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.