I am just curious here ... because we seem to have fedpkg-minimal and rhpkg-simple at least. And I've seen centpkg-minimal project too.
fedpkg-minimal
rhpkg-simple
centpkg-minimal
For the buildSRPMFromSCM task, e.g. Fedora Koji seems to do the git clone and git checkout on builder host, while the actual download from dist-git lookaside cache is done inside chroot, using fedpkg srpm command.
git clone
git checkout
fedpkg srpm
I was thinking about merging all the *pkg-{minimal,simple} packages into one, and providing some trivial configurable alternative good for everyone -- while keeping the benefit of minimal runtime dependencies (shell only, depending only on curl) .... but it seems to me that we could simply run the fedpkg srpm command on host (builder) and we wouldn't have to care about the additional (e.g. python) dependencies.
Also, doing such things on host, I bet it would simplify the chroot bootstrapping for new architectures, etc.
Does anyone know what is the original (or even actual) reason for this?
Yes, rhpkg is not the only mechanism which creates srpm. Generally you define SRPM-creation mechanism in builder configuration and it depends on concrete SCM. So, e.g. while majority uses distgit+rhpkg-simple, others can use even non-git SCM but they definitely can have different tools which needs to be run on source code before SRPM is created (from last months it is autobumping of release in fedora or selecting from multiple spec files in repo).
More from historical reasons (CVS/SVN fedora/rhel repos) is there possibility to use "common" subdirectory which can contain some scripts for handling the srpm. But this is another way how to "merge" *pkg-simple
Another reason is that we want to create srpm with tools from buildroot not from builder environment. This is of course (probably) the most important issue. We want to have same srpm if we're building on rhel5 builder or if we are building on rhel8 builder. So only way to use same rpm binary, etc is to do everything inside the buildroot.
Detail are here: https://docs.pagure.org/koji/server_howto/#source-control-configuration
Metadata Update from @tkopecek: - Custom field Size adjusted to None
Metadata Update from @tkopecek: - Issue tagged with: discussion
Thanks for the references.
Yes, rhpkg is not the only mechanism which creates srpm. Generally you define SRPM-creation mechanism in builder configuration and it depends on concrete SCM. So, e.g. while majority uses distgit+rhpkg-simple, others can use even non-git SCM but they definitely can have different tools which needs to be run on source code before SRPM is created
That's right, but would it be acceptable to run those commands directly on host, instead of in target chroot? I mean all those commands executed before the actual rpmbuild -bs call (which needs to stay executed in buildroot)? I'm trying to get some counter example, that would be very helpful.
rpmbuild -bs
More from historical reasons (CVS/SVN fedora/rhel repos) is there possibility to use "common" subdirectory
How Koji achieved that the common directory was available in buildroot?
common
Yeah, that's we want to use rpmbuild -bs from buildroot (no doubts, this is desired since RPMs from newer systems might not be readable by older systems..).
The only thing implemented in fedpkg srpm though seems to be to download sources based on the pwd basename + sources file. Moving that responsibility to host looks like no-trouble (it shouldn't cause differences in the final source RPM if we use buildroot for -bs).
pwd
-bs
From the documentation, it seems that we can not use too much imagination in source_cmd option anyway -- because we can not specify additional source RPM build requirements there (if we e.g. wanted to use python3 for some additional spec file tweaks, e.g. auto-bumping).
source_cmd
Ad "merging *pkg-simple" packages idea ... The more I think about this, the more I tend to think that we should move the responsibility to Mock and rather stop using those packages. Rather spending our time on merging them.
I mean, we use mock anyway (Koji and Copr), and it would be pretty trivial to teach Mock how to get sources from dist-git (either directly, or through python-rpkg).
Simply put, fedpkg srpm is building something that needs to be done reliably and not subject to the variations of the host os.
Why fedpkg srpm (fedpkg-simple) installed on builder host can not work reliably? What host variations do you have in mind now? At least we seem to rely on host's git-core stack to work reliably...
@praiskup git work as is. rpm is very dependent on version and features compiled in (e.g. in last year we had a problems with new zstd compression and changed checksum algorithm). drpm built on current fedora builder can't be opened by rhel6/7 rpm, etc.
Again, I'm not proposing to use rpm from host! Only the fedpkg-minimal thingy. We need to use rpmbuild -bs from buildroot, not host. And that's what Koji uses mock for anyway right now, or?
rpm
Nb fedpkg srpm (I didn't underline it so far, so there comes the misunderstanding... maybe) doesn't actually create the source RPM (contrary to how it looks, ...). It only downloads the source files from the dist-git lookaside cache. This particular functionality could be done on host. The srpm argument is actually ignored, from what I remember.
srpm
This issue has been migrated to Fedora Forge: https://forge.fedoraproject.org/koji/koji/issues/2432
Please continue any further discussion there.