fedora-project-config
Zuul CI Bot (Fabien Boucher) committed 15 days ago
fedora-project-config
Software Factory configuration for the Fedora project.
Here are configured:
- resources/: Git repositories to be listened for events by Fedora's Zuul.
- zuul.d/_pipelines.yaml: Zuul pipelines for the Fedora's Zuul jobs.
- playbooks/: Ansible playbooks used by some Fedora's Zuul jobs.
- tools/: Addtionnal tooling.
- FZCI.dhall: Dhall description of Fedora Zuul resources to be used to render Fedora's Zuul jobs configuration.
The Zuul CI configuration for Fedora is managed through 3 repositories:
fedora-project-config: this one.fedora-zuul-jobs-config: A Zuul config repository to host sensitive Zuul config (e.g. jobs relying on secrets).fedora-zuul-jobs: A Zuul untrusted repository to host non sensitive Zuul config.
Most of the Zuul config for Fedora is managed via the Dhall language. The rendering of the dhall program produces YAML files to be used by Zuul. YAML files are generated into the three repositories described above.
How to add / maintain your packages list
As a package maintainer who wishes to benefit from the Zuul CI, you need to add your packages to resources/fedora-distgits.yaml.
You can add them manually or use the zuul-config-generator. It generates a new config file containing all missing packages of a given maintaner or group.
Replace the existing resources/fedora-distgits.yaml with the newly generated file and submit a Pull Request to this repository.
Prepare your workspace to edit Fedora Zuul jobs config
Install required tools
dhall-to-yaml and zuulfmt are needed to render the Zuul configuration.
To install dhall-to-yaml runs (or adapt to your working environment)
dnf install -y dhall-json
To install zuulfmt runs (or adapt to your working environment)
python -mpip install --user zuulfmt
It is adviced to install the Dhall lang support for your IDE:
Clone/Update jobs repositories
Run the following command to install (and/or reset to the last origin/HEAD) extra repositories.
This prepares fedora-zuul-jobs-config and fedora-zuul-jobs checkout in the parent directory.
make update-jobs-repos
Then:
- Fork both repositories in your pagure.io user namesapce
- Use
git remote add fork ssh://git@pagure.io/forks/<username>/<repo>.git
How to update Zuul jobs config
First, ensure repositories are in the right state (all commands below assume working from the master branch):
git checkout master && git fetch origin && git reset --hard origin/master make update-jobs-repos
Second, add the required changes.
Then run the rendering.
[fedora@localhost fedora-project-config]$ make render /bin/dhall-to-yaml /home/fedora/bin/zuulfmt Rendering FZCI.dhall/fedora-zuul-jobs-config/jobs.dhall in ../fedora-zuul-jobs-config/zuul.d/jobs.yaml Rendering FZCI.dhall/fedora-zuul-jobs/jobs.dhall in ../fedora-zuul-jobs/zuul.d/jobs.yaml Rendering FZCI.dhall/fedora-zuul-jobs/nodesets.dhall in ../fedora-zuul-jobs/zuul.d/nodesets.yaml Rendering FZCI.dhall/fedora-zuul-jobs/templates.dhall in ../fedora-zuul-jobs/zuul.d/templates.yaml
In case of error, update your changes then run again the rendering.
Finally, create Pull Requests for each repositories that got an update from the rendering.
For instance:
make check-updates # For each repo where this shows a diff then run: (here a change in fedora-zuul-jobs) pushd ../fedora-zuul-jobs git status git add -A . git commit -m"Add change for ..." git push fork master:pr-branch-on-fork
And open the PR on pagure.io.
HOWTOs
Add support for a Fedora Branch
Adding new branch definition
- Edit file branches.dhall in order to add the new-branch
- The type checker will help you to fill the places where the new-branch handling must be done.
If you don't have the support of Dhall in your IDE, you might want to check your edit with the type checker:
dhall <<< './FZCI.dhall/Branches.dhall'
Update jobs from fedora-zuul-jobs-config
- Edit fedora-zuul-jobs-config/jobs.dhall to handle the new-branch. Again the type checker will help you.
If the type checker does not complain then nothing needs to be done into that file.
Then run:
make render
Update jobs from fedora-zuul-jobs
- Edit fedora-zuul-jobs/jobs.dhall in order to ensure jobs will trigger in the new-branch. Again the type checker will help you.
Then run:
make render
Push changes
When edits are done on those repos then commits and push changes (as PR) then wait for CI result.
Update the latest nodeset definition
When a new fedora release is available, the fedora-latest-* nodesets need to be updated.
Two changes are required:
- The softwarefactory-project.io/config Nodepool service.
- The fedora-zuul-jobs Zuul configuration.
Here are the required changes and example changes from a previous update.
Add new Nodepool labels
Wait for the infra team to provide the new labels. For examples, here are the changes for f38:
- https://softwarefactory-project.io/r/c/containers/+/28351 - Create a new container image (and the new quay image needs to be switched to public).
- https://softwarefactory-project.io/r/c/config/+/27464 - Create a new cloud image.
- https://softwarefactory-project.io/r/c/config/+/28352 - Register the new container image.
Update Zuul latest definition
Update the latest-version value in Nodesets.dhall. For example, here is the change for f38:
Run the make render command and update the fedora-zuul-jobs project. For example, here is the change for f38: