From dcffe931d19b67793df3dd096b741f18d6a21660 Mon Sep 17 00:00:00 2001 From: Merlin Mathesius Date: Nov 26 2018 21:44:55 +0000 Subject: [PATCH 1/2] Add space for module scratch builds --- diff --git a/module-scratch-builds/README.md b/module-scratch-builds/README.md new file mode 100644 index 0000000..7626b1b --- /dev/null +++ b/module-scratch-builds/README.md @@ -0,0 +1,7 @@ +# Module Scratch Builds + +**Tracked in [#15 Module scratch builds](https://tree.taiga.io/project/modularity-wg/epic/15).** + +* **Proposal for Module Scratch Builds** — A technical proposal describing how module scratch builds should work. + * **Tracker:** [#51 Draft a proposal for module scratch builds](https://tree.taiga.io/project/modularity-wg/us/51) + * **Doc:** [Proposal for Module Scratch Builds](https://pagure.io/modularity/working-documents/blob/master/f/module-scratch-builds/module-scratch-build-proposal.md) diff --git a/module-scratch-builds/module-scratch-build-proposal.md b/module-scratch-builds/module-scratch-build-proposal.md new file mode 100644 index 0000000..4c532f5 --- /dev/null +++ b/module-scratch-builds/module-scratch-build-proposal.md @@ -0,0 +1,4 @@ +# Proposal for Module Scratch Builds + +**Tracked in [#51 Draft a proposal for module scratch builds](https://tree.taiga.io/project/modularity-wg/us/51) + From d21e376206d1e26ebd43dc327ce4a8ef2f89c3df Mon Sep 17 00:00:00 2001 From: Merlin Mathesius Date: Nov 30 2018 16:26:19 +0000 Subject: [PATCH 2/2] Add content to module scratch build proposal --- diff --git a/module-scratch-builds/module-scratch-build-proposal.md b/module-scratch-builds/module-scratch-build-proposal.md index 4c532f5..57d2f64 100644 --- a/module-scratch-builds/module-scratch-build-proposal.md +++ b/module-scratch-builds/module-scratch-build-proposal.md @@ -1,4 +1,44 @@ # Proposal for Module Scratch Builds -**Tracked in [#51 Draft a proposal for module scratch builds](https://tree.taiga.io/project/modularity-wg/us/51) +**Tracked in [#51 Draft a proposal for module scratch builds](https://tree.taiga.io/project/modularity-wg/us/51)** + +_Credit to Jan Kaluža (@jkaluza) for the content of this proposal._ + +The goal of this document is to propose a way to implement module scratch-builds as defined in [Taiga #15 Module scratch builds](https://tree.taiga.io/project/modularity-wg/epic/15). + +Currently, it focuses more on the build-side (MBS side) of the problem and omits the fedpkg user-experience, but others may feel free to extend this document to also cover this part. + +This document also briefly mentions alternative ways which have been discussed as part of a meeting on this topic. + +## Proposed implementation + +1. New command to run module scratch-build is implemented. This will do following: + * `rpkg` (and, by extension, `fedpkg`) will be modified to recognize new `module-scratch-build` command as well as `--scratch` option to `module-build` command—which are synonymous. + * Allow specifying SRPMs of components which will override the default source from which to get the component in the modulemd file. + * Uploads the SRPMs to Koji the same way as it's done for a normal SRPM scratch-build. + * Submits the module build to MBS including the modulemd file, paths to uploaded SRPM files in the Koji upload directory, and a flag that this build is supposed to be a scratch build. +2. MBS will handle a scratch build like a normal module build, but with following differences: + * MBS will allow a full modulemd string as its input, as well as a list of custom SRPMs which should be used when building the module. + * For components which are overridden in the MBS input by a custom SRPM, it will build this custom SRPM instead of building the component from dist-git. + * The Koji tag for scratch-builds will have special prefix - for example "module-scratch-" and special unique suffix, so it is possible to build the same NSVC twice. + * The dist tag of the component builds in the module scratch-build will have a special prefix/suffix that is uniquely generated, distinguishing the RPMs belonging to this module scratch-build. This makes it is possible to build the same NSVC twice. + * The Koji content-generator build won't be generated for a scratch-build. + * Non-scratch-builds cannot reuse components from a scratch-build. + * Both scratch and non-scratch builds cannot buildrequire a scratch build. +3. Koji-gc will be configured to remove module scratch build Koji tags of a yet to be determined age as well as scratch components tagged in these Koji tags. Note that a scratch-build can reuse non-scratch-build packages, so these must not be removed. +4. Koji hub policy should be modified to disallow the tagging of these builds into any candidate or release tags. + * It was suggested that the Koji hub policy mechanism is not strong enough to protect us here: that someone can just untag one of these builds, and then re-tag it wherever they want. This is not true. Policies can be applied based on what tag a build is currently in, but they can also be applied based on the buildtag for the build. Use that. + +## Alternative way #1: Using RPM scratch-builds instead of real builds + +An alternative way to implement module scratch builds is to run component builds as Koji scratch-builds. This has been considered but not chosen as the proposed way because of following issues: + +* Koji can generate repository which can be used in an RPM buildroot only from Koji builds tagged into some Koji tag, but RPM scratch build cannot be tagged into Koji tags. This would need major Koji changes. We would need this feature to build components which depend on previously built components in the same module. + +## Alternative way #2: Generating the repository with RPM scratch-builds outside of Koji + +To workaround the limitation of the previous alternative solution, it has been proposed to generate the RPM repository with scratch-builds outside of Koji, and generate a Koji tag which would point to that external repository. + +There is however no tool which would manage the creation of an RPM repository with a list of RPM scratch-builds. The closest service which could allow that would be ODCS, but it uses Pungi as a backend which is not designed to do this task, and implementing this feature would again result in non-trivial changes to its code. +