fedora-ci / general

#495 Zuul check-for-arches has wrong results, builds noarch packages on multiple arches or vice versa
Closed by fbo. Opened by churchyard.

Closed
Reopen Issue

Example noarch packages built on many arches:

https://src.fedoraproject.org/rpms/python-scripttest/pull-request/3
https://src.fedoraproject.org/rpms/xonsh/pull-request/8

Example arch package not built on many arches:

https://src.fedoraproject.org/rpms/scipy/pull-request/46

For some packages, this works, e.g. python3.14 is correctly built on all arches :/

This is the exact playbook we run as a check-for-arches job:

https://pagure.io/fedora-zuul-jobs-config/blob/master/f/playbooks/rpm/check-for-arches.yaml#_4

Can you suggest a more reliable way to detect noarch packages? Whatever script we can run which gives yes or no answer?

 
[python-scripttest (rawhide %)]$  rpm --specfile *.spec
python-scripttest-1.3.0-1.fc39.src
python3-scripttest-1.3.0-1.fc39.src
[xonsh (rawhide)]$ rpm --specfile *.spec 
xonsh-0.18.3-1.fc39.src
[scipy (rawhide %)]$ rpm --specfile *.spec 
scipy-1.11.3-15.fc39.src
python3-scipy-1.11.3-15.fc39.src
python3-scipy-tests-1.11.3-15.fc39.src
scipy-debuginfo-1.11.3-15.fc39.src
scipy-debugsource-1.11.3-15.fc39.src

heh, this is unexpected. I could swear this used to return noarch/x86_64 packages.

However, the results for all the packages are the same. Which is not consistent with the behavior of Zuul.

On rawhide, I get:

[root@c2847565696b python-scripttest]# rpm --specfile *.spec
python-scripttest-1.3.0-1.fc42.noarch
python3-scripttest-1.3.0-1.fc42.noarch
[root@c2847565696b scipy]# rpm --specfile *.spec
scipy-1.11.3-15.fc42.x86_64
python3-scipy-1.11.3-15.fc42.x86_64
python3-scipy-tests-1.11.3-15.fc42.x86_64

I reported this to RPM: https://github.com/rpm-software-management/rpm/issues/3402

We are currently using Fedora 40 workers in Zuul, and it seems we indeed get src:

https://fedora.softwarefactory-project.io/zuul/build/f45c583ddc5540dba72ccdb67f5b06cf/log/job-output.txt

2024-10-18 12:49:29.865306 | TASK [Check for noarch]
2024-10-18 12:49:34.626715 | container | src
2024-10-18 12:49:34.627282 | container | src
2024-10-18 12:49:35.560298 | container | ok: Runtime: 0:00:00.219575

Interestingly, for python3.14, it got:

2024-10-18 19:28:27.607270 | TASK [Check for noarch]
2024-10-18 19:28:32.010350 | container | src
2024-10-18 19:28:32.010753 | container | src
2024-10-18 19:28:32.010816 | container | src
2024-10-18 19:28:32.010948 | container | src
2024-10-18 19:28:32.010965 | container | src
2024-10-18 19:28:32.010976 | container | src
2024-10-18 19:28:32.010987 | container | src
2024-10-18 19:28:32.010997 | container | src
2024-10-18 19:28:32.011008 | container | src
2024-10-18 19:28:32.011018 | container | src
2024-10-18 19:28:32.011028 | container | src

yet it was built for all architectures. https://src.fedoraproject.org/rpms/python3.14/pull-request/8

I see what is going on.

All (sub)packages are always reported as src, which generates lines in stdout, hence everything is built on all arches. Except for packages that cannot be parsed by old RPM (which si the scipy case), where the stdout output is empty and hence the playbook reports the package is noarch.

There are two problems:

  1. https://github.com/rpm-software-management/rpm/issues/3402
  2. when the command rpm --specfile command fails, the reasonable default would be to report "not noarch" rather than "noarch"
Edited by churchyard

Yes, the logic of the check, as i understand it:

  • show all lines, filter out anything with noarch, and if there is something left (anything, src included) , then build for all arches (child_jobs: "{{ arch_jobs }}")

So empty output is treated as noarch.
Non-empty src output is treated as "all arches"

Ultimately the goal of the job is to assign value to child_jobs dictionary. It should be a list of build jobs to run.

The current playbook defines it as "child_jobs = arch_jobs if not noarch_package", where arch_jobs is just a static predefined dictionary with all arch jobs from https://pagure.io/fedora-zuul-jobs-config/blob/master/f/zuul.d/jobs.yaml#_59

But we can put any logic there as long as the child_jobs variable is set to contain the names of the jobs in the end.

Assuming the rpm got fixed, we can do something like 

- hosts: all
  tasks:
    - name: Get all mentioned architectures
      command: sh -c "rpm --specfile *.spec 2> stderr.log | rev | cut -d'.' -f 1 | rev | sort | uniq"
      register: arches
      failed_when: false
      args:
        chdir: "{{ zuul.project.src_dir }}"
  // Now "{{arches.stdout_lines }}" is a list like ['noarch', 'x86_64', 's390']
   - name: Turn the list of arches into the list of arch-dependent jobs
     ansible.builtin.set_fact:
        current_arch_jobs: "{{ arches.stdout_lines | map('regex_replace', '^', 'rpm-scratch-build-') | list }}"
  // Now "{{current_arch_jobs }}" is a list like ['rpm-scratch-build-noarch', ''rpm-scratch-build-x86_64', 'rpm-scratch-build-s390']
   - name: Filter jobs to see only available arch-dependent jobs
     ansible.builtin.set_fact:
        current_available_jobs: "{{ current_arch_jobs | intersect(arch_jobs) }}"
    - zuul_return:
        data:
          zuul:
            child_jobs:
              - noop
      when: (not check_for_arches_ignore) and current_available_jobs == []
    - zuul_return:
        data:
          zuul:
            child_jobs: "{{ current_available_jobs }}"
      when: (not check_for_arches_ignore) and current_available_jobs != []
    - zuul_return:
        data:
          zuul:
            child_jobs: "{{ arch_jobs }}"
      when: check_for_arches_ignore

I like that proposal, also inline comments are really valuable in that case. Furthermore handling the excludeArch might be also a good addition (see https://pagure.io/fedora-ci/general/issue/496).

See also https://pagure.io/fedora-ci/general/issue/496#comment-941399

https://pagure.io/fedora-ci/general/issue/496#comment-946270

Closing the ticket please re-open if needed.

Metadata Update from @fbo:
- Issue status updated to: Closed (was: Open)
Metadata
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.