#8521 Speed up ipa-server-install
Closed: fixed by antorres. Opened by cheimes.

Request for enhancement

ipa-server-install and ipa-replica-install take a considerable time to complete. On a fast server with plenty resources a server installation takes about 5 minutes. On a slower system with less resources installation can take twice as long or even longer.

My experiments have shown that it is possible to speed up installation with a series of small modifications. There are several low hanging fruits. Each change contributes a couple of seconds. Combined I see runtime reduction from roughly 5 minutes down to 3:30 minutes for ipa-server-install.

  • start / restart services less often (DS ~ 4s, Dogtag 11 to 50s)
  • tighten poll/sleep loops with lower sleep intervalls (5s to 0.5s)
  • remove steps or workarounds that are no longer necessary
  • combine steps

Changes


master:

  • aa67177f4cbea0f51574e890c3a8543ff603d5d2 Add helper for poll/sleep loops with timeout
  • b79191f7107fbc1890631f4c65be48b646798131 Faster certmonger wait_for_request()
  • 1921d33d41f5b0e1632f7876a0acbde5462ea8be Drop unused extended sleep feature from Sleeper

Rob wrote:

Is migrate_profiles_ldap() needed at all then? It's executed as part of the upgrade. Speeding that up would be nice too.

From IRC discussion about LDAP profile migration during upgrade:

18:04 < ftweedal> cheimes: a thought about optimising the certprofile import during upgrade.  Not going to bother 
                  with pseudocode or anything, but we can probably conclude that if *any* profiles are already in 
                  LDAP, then *all* are in LDAP, and short circuit the whole thing.
18:05 < ftweedal> cheimes: on a per-host basis, we could also set the "we did this step" flag so that the skip gets 
                  skipped on future upgrades and we don't have to contact Dogtag at all
18:06 < ftweedal> cheimes: for the "freeipa included profiles" we probably do want to continue managing them in the 
                  current way.
18:06 < ftweedal> or at least, that is something we can look at later - there are only a few of them so it is not as
                  a big an issue

master:

  • 3ab3ed5ff29bb389c28feb1d5398e76d4b04926c Retry chronyc waitsync only once

master:

  • 37a0af6a8cc716acd589b6295039af8ae20720fa Remove root-autobind configuration
  • daec8049610ad7bbe0718276d2e7d6ffc1a7f66e Remove magic sleep from create_index_task
  • 9eccaf62697f5281729c7d0f38265c66a7264e25 Skip offline dse.ldif patching by default

master:

  • 8882680ee167c535e7539fe0452892aae617aac3 Dogtag: Remove set_audit_renewal step
  • 942fe07eb53dbbacb33352bbb67039abf73b2544 Spawn PKI: Execute more steps early

master:

  • 731c5b211041ee18940082b29e6a21b3f084fc57 Lookup ipa-ca record with NSS

ipa-4-8:

  • 81cf1db50e0b3431c07d206b7b774679cb163e03 Lookup ipa-ca record with NSS

master:

  • a9d34c8e66590c6ddb18a227fa0201cacdc7d72d Speed up cainstance.migrate_profiles_to_ldap
  • fa58071221bfb37159db6d6dc4f3bcfd088d80b1 Reuse main LDAP connection

Some numbers from a test installation on 1minutetip m1.large instance (8 GB RAM, 4x Intel Xeon Processor 2.7 GHz)

# rpm -qa freeipa-server
freeipa-server-4.8.10-5.fc32.x86_64
# time ipa-server-install -p Secret123 -a Secret123 -r IPA.EXAMPLE -n ipa.example --no-ntp -U
...
real    5m12,632s
user    0m31,968s
sys     0m6,665s
# curl http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/a26d8e50-11e0-11eb-b85e-fa163ee3281e/rpms/freeipa-prci.repo > /etc/yum.repos.d/freeipa-prci.repo
# dnf update -y
# rpm -qa freeipa-server
freeipa-client-4.9.0.dev-0.fc32.x86_64
# time ipa-server-install -p Secret123 -a Secret123 -r IPA.EXAMPLE -n ipa.example --no-ntp -U
...
real    3m20,871s
user    0m29,771s
sys     0m6,821s

On 1MT the installer spends a considerable amount of time waiting on DNS. With ipa-server-install -p Secret123 -a Secret123 -r IPA.EXAMPLE -n ipa.example --no-ntp --no-host-dns --setup-dns --auto-forwarders --no-dnssec-validation -U speedup is down to about 1 minute:

freeipa-server-4.8.10-5.fc32.x86_64
real    4m19,641s
user    0m33,342s
sys     0m6,575s
freeipa-client-4.9.0.dev-0.fc32.x86_64
real    3m18,411s
user    0m28,540s
sys     0m6,327s

Closing as all linked PRs are already merged.

Metadata Update from @antorres:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata