releng

#8968 enable epel8 modules in bodhi
Closed: Fixed by merlinm. Opened by merlinm.

Closed: Fixed
Reopen Issue

  • Describe the issue
    Now that modules can be built for the el8 platform, they need to be included in the bodhi release process.

  • When do you need this? (YYYY/MM/DD)
    ASAP

  • When is this no longer needed or useful? (YYYY/MM/DD)
    A long time from now.

  • If we cannot complete your request, what is the impact?
    EPEL8 module builds are not usable.

@merlinm @kevin

Shell script to add tags (removed the perms on the tag since its only staging and autosign doesn't work properly)

release=epel8
modular_release=${release}-modular
stg-koji add-tag --parent "${release}" "${release}-modularity"
stg-koji add-tag "${modular_release}"
stg-koji edit-tag -x mock.package_manager=dnf "${modular_release}"
stg-koji add-tag --parent "${modular_release}" "${modular_release}-updates"
stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-updates-candidate"
stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-updates-testing"
stg-koji add-tag --parent "${modular_release}-updates-testing" "${modular_release}-updates-testing-pending"
stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-updates-pending"
stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-override"
stg-koji add-tag --parent "${modular_release}-updates-testing-pending" "${modular_release}-signing-pending"
stg-koji add-tag --parent "${modular_release}-updates"  "${modular_release}-pending"
#stg-koji edit-tag --perm=fedora-override "${modular_release}-override"
#stg-koji edit-tag --perm=admin "${modular_release}-updates"
#stg-koji edit-tag --perm=admin "${modular_release}-updates-testing"
#stg-koji edit-tag --perm=autosign "${modular_release}-updates-testing-pending"
#stg-koji edit-tag --perm=admin "${modular_release}-updates-pending"
#stg-koji edit-tag --perm=autosign "${modular_release}-signing-pending"
#stg-koji edit-tag --perm=autosign "${modular_release}"
release=epel8-playground
modular_release=${release}-modular
stg-koji add-tag --parent "${release}" "${release}-modularity"
stg-koji add-tag "${modular_release}"
stg-koji edit-tag -x mock.package_manager=dnf "${modular_release}"
stg-koji add-tag --parent "${modular_release}"  "${modular_release}-updates-candidate"

Infra ansible diff for bodhi epel8 composes:

diff --git a/roles/bodhi2/backend/templates/pungi.module.conf.j2 b/roles/bodhi2/backend/templates/pungi.module.conf.j2
index d6571fb..1150e73 100644
--- a/roles/bodhi2/backend/templates/pungi.module.conf.j2
+++ b/roles/bodhi2/backend/templates/pungi.module.conf.j2
@@ -20,14 +20,22 @@ sigkeys = [
        '3c3359c4',
 [% endif %]
 {% if env == "staging" %}
-       None
+    [% elif release.version_int == 8 %]
+       '2f86d6a1',
+    [% endif %]
 {% endif %}
 ]
 module_defaults_dir = {
     'scm': 'git',
     'repo': 'https://pagure.io/releng/fedora-module-defaults.git',
-    'branch': 'f[[ release.version_int ]]',
+    {% if env == "staging" %}
+        [% elif release.version_int == 8 %]
+            'branch': 'epel[[ release.version_int ]]',
+        [% endif %]
+    {% else %}
+        'branch': 'f[[ release.version_int ]]',
+    {% endif %}
     'dir': '.'
 }

epel8-playground module composes can be enabled by using
https://pagure.io/fork/mohanboddu/pungi-fedora/diff/master..epel8-playground-modules-stg
For stg, we can use the one in my fork directly
Edited by mohanboddu

Updated diff for bodhi:

diff --git a/roles/bodhi2/backend/templates/pungi.module.conf.j2 b/roles/bodhi2/backend/templates/pungi.module.conf.j2
index d6571fb..54e4543 100644
--- a/roles/bodhi2/backend/templates/pungi.module.conf.j2
+++ b/roles/bodhi2/backend/templates/pungi.module.conf.j2
@@ -20,14 +20,22 @@ sigkeys = [
        '3c3359c4',
 [% endif %]
 {% if env == "staging" %}
-       None
+    [% if release.version_int == 8 %]
+       '2f86d6a1',
+    [% endif %]
 {% endif %}
 ]
 module_defaults_dir = {
     'scm': 'git',
     'repo': 'https://pagure.io/releng/fedora-module-defaults.git',
-    'branch': 'f[[ release.version_int ]]',
+    {% if env == "staging" %}
+        [% if release.version_int == 8 %]
+            'branch': 'el[[ release.version_int ]]',
+        [% endif %]
+    {% else %}
+        'branch': 'f[[ release.version_int ]]',
+    {% endif %}
     'dir': '.'
 }

epel8-playground module composes can be enabled by using
https://pagure.io/fork/mohanboddu/pungi-fedora/diff/master..epel8-playground-modules-stg
For stg, we can use the one in my fork directly

Just for reference, https://pagure.io/fork/mohanboddu/pungi-fedora/diff/epel8-playground..epel8-playground-modules-stg is a more appropriate diff to examine.

Just need to figure out how https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/mbs/common/templates/config.py#n177 and https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/mbs/common/templates/config.py#n178 works with epel8 tags.

Final script for tags:

release=epel8
modular_release=${release}-modular
echo $release
echo $modular_release
stg-koji add-tag --parent "${release}" "${release}-modularity"
stg-koji add-tag "${modular_release}"
stg-koji edit-tag -x mock.package_manager=dnf "${modular_release}"
stg-koji add-tag --parent "${modular_release}" "${modular_release}-updates"
stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-updates-candidate"
stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-updates-testing"
stg-koji add-tag --parent "${modular_release}-updates-testing" "${modular_release}-updates-testing-pending"
stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-updates-pending"
stg-koji add-tag --parent "${modular_release}-updates" "${modular_release}-override"
stg-koji add-tag --parent "${modular_release}-updates-testing-pending" "${modular_release}-signing-pending"
stg-koji add-tag --parent "${modular_release}-updates"  "${modular_release}-pending"
stg-koji edit-tag --perm=fedora-override "${modular_release}-override"
stg-koji edit-tag --perm=admin "${modular_release}-updates"
stg-koji edit-tag --perm=admin "${modular_release}-updates-testing"
stg-koji edit-tag --perm=autosign "${modular_release}-updates-testing-pending"
stg-koji edit-tag --perm=admin "${modular_release}-updates-pending"
stg-koji edit-tag --perm=autosign "${modular_release}-signing-pending"
stg-koji edit-tag --perm=autosign "${modular_release}"
release=epel8-playground
modular_release=${release}-modular
echo $release
echo $modular_release
stg-koji add-tag --parent "${release}" "${release}-modularity"
stg-koji add-tag "${modular_release}"
stg-koji edit-tag -x mock.package_manager=dnf "${modular_release}"
stg-koji add-tag --parent "${modular_release}"  "${modular_release}-updates-candidate"
stg-koji edit-tag --perm=autosign "${modular_release}"

Diff for infra ansible:

I am not sure what are the keys in staging right now, I can check once my sigul starts working.

diff --git a/roles/bodhi2/backend/templates/pungi.module.conf.j2 b/roles/bodhi2/backend/templates/pungi.module.conf.j2
index d6571fb..8307069 100644
--- a/roles/bodhi2/backend/templates/pungi.module.conf.j2
+++ b/roles/bodhi2/backend/templates/pungi.module.conf.j2
@@ -20,14 +20,24 @@ sigkeys = [
        '3c3359c4',
 [% endif %]
 {% if env == "staging" %}
-       None
+    [% if release.version_int == 8 %]
+       '2f86d6a1',
+    [% endif %]
 {% endif %}
 ]
 module_defaults_dir = {
     'scm': 'git',
     'repo': 'https://pagure.io/releng/fedora-module-defaults.git',
-    'branch': 'f[[ release.version_int ]]',
+    {% if env == "staging" %}
+        [% if release.version_int == 8 %]
+            'branch': 'el[[ release.version_int ]]',
+        [% else %]
+            'branch': 'f[[ release.version_int ]]',
+        [% endif %]
+    {% else %}
+        'branch': 'f[[ release.version_int ]]',
+    {% endif %}
     'dir': '.'
 }
diff --git a/roles/robosignatory/templates/robosignatory.toml.j2 b/roles/robosignatory/templates/robosignatory.toml.j2
index a06043d..518cd2f 100644
--- a/roles/robosignatory/templates/robosignatory.toml.j2
+++ b/roles/robosignatory/templates/robosignatory.toml.j2
@@ -257,6 +257,21 @@ handlers = ["console"]
             keyid = "{{ (env == 'production')|ternary('3c3359c4', 'd300e724') }}"
             type = "modular"
+            #epel8 modular tags
+            [[consumer_config.koji_instances.primary.tags]]
+            from = "epel8-modular-signing-pending"
+            to = "epel8-modular-updates-testing-pending"
+            key = "{{ (env == 'production')|ternary('epel-8', 'testkey') }}"
+            keyid = "{{ (env == 'production')|ternary('2f86d6a1', 'd300e724') }}"
+            type = "modular"
+
+            [[consumer_config.koji_instances.primary.tags]]
+            from = "epel8-playground-modular-updates-candidate"
+            to = "epel8-playground-modular"
+            key = "{{ (env == 'production')|ternary('epel-8', 'testkey') }}"
+            keyid = "{{ (env == 'production')|ternary('2f86d6a1', 'd300e724') }}"
+            type = "modular"
+            
             # Gated bodhi updates
             [[consumer_config.koji_instances.primary.tags]]

@mohanboddu,

Just need to figure out how https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/mbs/common/templates/config.py#n177 and https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/mbs/common/templates/config.py#n178 works with epel8 tags.

Based on my interpretation of the logic in get_content_generator_build_koji_tag() at https://pagure.io/fm-orchestrator/blob/master/f/module_build_service/scheduler/handlers/modules.py#_268, the MBS will use KOJI_CG_BUILD_TAG_TEMPLATE (config.py#n177) as the template for the Koji tag--filling in the stream of the base module. Thus, for an el8 base module, it will use tag el8-modular-updates-candidate. If it can't figure out a base module--which AFAIK should never happen, it will fall back to KOJI_CG_DEFAULT_BUILD_TAG (config.py#n178) and use tag modular-updates-candidate.

I don't think there's currently a way to get an el8 base module to map to tag epel8-modular-updates-candidate while keeping Fedora modules mapping to f##-modular-updates-candidate.

Perhaps @mprahl or @jkaluza can confirm or deny my analysis.

@merlinm Oh, shoot, I looked at it and changed the signing tags for playground to use -updates-candidate instead of -signing-pending, but I didn't realize the epel8 vs el8 problem :sob:

@mohanboddu What is the path forward with this? How are you expecting MBS to tag modules built for el8 and el8_playground? Is there a problem with MBS tagging module builds as el8-modular-updates-candidate rather than epel8-modular-updates-candidate? (Likewise for el8_playground-modular-updates-candidate vs. epel8-playground-modular-updates-candidate) Can't that just be adjusted in roles/robosignatory/templates/robosignatory.toml.j2?

So, I changed the candidate tags to el8-{playground-}updates-candidate.

$ stg-koji edit-tag epel8-playground-modular-updates-candidate --rename el8_playground-modular-updates-candidate
$ stg-koji edit-tag epel8-modular-updates-candidate --rename el8-modular-updates-candidate

And to create the bodhi epel 8 modular releases in stage:

bodhi releases create --name EPEL-8M --long-name "Epel 8 Modular" --id-prefix FEDORA-EPEL-MODULAR --version 8 --branch epel8m --dist-tag epel8-modular --stable-tag epel8-modular-updates --testing-tag epel8-modular-updates-testing --candidate-tag el8-modular-updates-candidate --pending-stable-tag epel8-modular-updates-pending --pending-testing-tag epel8-modular-updates-testing-pending --pending-signing-tag epel8-modular-signing-pending --override-tag epel8-modular-override --state current --user mohanboddu --staging

And again, the diff from infra ansible:

diff --git a/roles/bodhi2/backend/templates/pungi.module.conf.j2 b/roles/bodhi2/backend/templates/pungi.module.conf.j2
index d6571fb..feeb320 100644
--- a/roles/bodhi2/backend/templates/pungi.module.conf.j2
+++ b/roles/bodhi2/backend/templates/pungi.module.conf.j2
@@ -20,14 +20,25 @@ sigkeys = [
        '3c3359c4',
 [% endif %]
 {% if env == "staging" %}
-       None
+    [% if release.version_int == 8 %]
+        'd300e724',
+    [% endif %]
 {% endif %}
 ]
 module_defaults_dir = {
     'scm': 'git',
-    'repo': 'https://pagure.io/releng/fedora-module-defaults.git',
-    'branch': 'f[[ release.version_int ]]',
+    {% if env == "staging" %}
+        'repo': 'https://pagure.io/modularity/fedora-stg-module-defaults.git',
+        [% if release.version_int == 8 %]
+            'branch': 'el[[ release.version_int ]]',
+        [% else %]
+            'branch': 'f[[ release.version_int ]]',
+        [% endif %]
+    {% else %}
+        'repo': 'https://pagure.io/releng/fedora-module-defaults.git',
+        'branch': 'f[[ release.version_int ]]',
+    {% endif %}
     'dir': '.'
 }
diff --git a/roles/robosignatory/templates/robosignatory.toml.j2 b/roles/robosignatory/templates/robosignatory.toml.j2
index 2df04d3..8126648 100644
--- a/roles/robosignatory/templates/robosignatory.toml.j2
+++ b/roles/robosignatory/templates/robosignatory.toml.j2
@@ -257,6 +257,21 @@ handlers = ["console"]
             keyid = "{{ (env == 'production')|ternary('3c3359c4', 'd300e724') }}"
             type = "modular"
+            #epel8 modular tags
+            [[consumer_config.koji_instances.primary.tags]]
+            from = "epel8-modular-signing-pending"
+            to = "epel8-modular-updates-testing-pending"
+            key = "{{ (env == 'production')|ternary('epel-8', 'testkey') }}"
+            keyid = "{{ (env == 'production')|ternary('2f86d6a1', 'd300e724') }}"
+            type = "modular"
+
+            [[consumer_config.koji_instances.primary.tags]]
+            from = "el8_playground-modular-updates-candidate"
+            to = "epel8-playground-modular"
+            key = "{{ (env == 'production')|ternary('epel-8', 'testkey') }}"
+            keyid = "{{ (env == 'production')|ternary('2f86d6a1', 'd300e724') }}"
+            type = "modular"
+
             # Gated bodhi updates
             [[consumer_config.koji_instances.primary.tags]]

@merlinm @kevin One last review and I ready to push the changes.
Edited by mohanboddu

@mohanboddu The koji tag will need to be el8_playground-modular-updates-candidate (rather than el8-playground-modular-updates-candidate) to exactly match platform:el8_playground. Likewise in roles/robosignatory/templates/robosignatory.toml.j2.

There's also a separate module defaults repo for staging. module_defaults_dir['repo'] (referenced in roles/bodhi2/backend/templates/pungi.module.conf.j2) should be set to https://pagure.io/modularity/fedora-stg-module-defaults.git in staging.

As far as I know, the rest looks good. Thanks!

Just updated the comment so that I will everything in one place.

The parts I kinda understand LGTM!

@mohanboddu I just noticed that https://pagure.io/fork/mohanboddu/pungi-fedora/blob/epel8-playground-modules-stg/f/epel8-playground.conf#_17 refers to the production modular defaults repo. Unless there's a straight forward way to create separate vs. production contents in epel8-playground.conf, we should be OK leaving it pointing at the production modular defaults. If it turns out to be a problem, we can create a custom staging configuration later.

:information_source: The above changes were deployed to staging on 2019-11-15. A modular el8 bodhi update attempt failed upon submit with the error described by https://github.com/fedora-infra/bodhi/issues/3779. Fix https://github.com/fedora-infra/bodhi/pull/3782 was applied and the bodhi update was submitted successfully.

Here is a patch to add the production EPEL 8 virtual platform module definitions to the Infra ansible repo.

Attached
0001-Adding-production-epel8-platform-module-definitions.patch

To move this production, in addition to re-implementing everything above, the following is also needed...

  • The tag structure described in https://pagure.io/releng/issue/8865 for staging also needs to be created in production.
  • Base module architecture tweaking may need to be done for the MBS production config as was done for staging by https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=cdff93b2ec15aecac81ec60da61ee1b7d7893098

@merlinm Is this now complete?

Yes!

And the production release is tracked in https://pagure.io/releng/issue/9091

Metadata Update from @merlinm:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)
Metadata
None
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.