This rather sounds like a request for chain build or bootstrapping support.

Having "limit" only as you requested it -- you wouldn't be able to control your builds correctly.. I mean, the fact that you need to control the order of the very first builds of each package doesn't mean that you always want this (once you have your project bootstrapped, you don't want to apply the limit).

Also, putting artificial bottleneck like this wouldn't help in future because we don't guarantee the order of builds anyways - the fact that it is ordered nowadays is mostly coincidence.

Right, I buried the full request in the middle of the paragraph, sorry:

I'd like to propose a repo configuration value that forces builds to be completed one at time, ordered by [submission] id.

So, chain build would be closest. This issue extends beyond the very first builds of packages as well. If it were just bootstrapping, I could get away with doing those manually, once.

Consider when we bump the version of JSS, then update PKI to dep on the new JSS (and, some feature in it)? If we submit those builds to a repo with an existing tree, it still won't build because PKI will dep on nvr+1, which is building in parallel to JSS.

@edewata proposed making the RFE be "add support for parsing the dependencies of the submitted jobs and building them in order based on that". This would be more complete (e.g., if both JSS and PKI are submitted but PKI doesn't strictly depend on this submitted JSS build, it could go in parallel, but if PKI deps on the new JSS NVR, it'll have to wait). I thought that limiting the project to only having one build in the build queue at a time would be easier for the COPR team to implement than parsing the RPM dependency tree of submitted jobs.

My 2c.

There aren't any guarantees that build with higher ID will be processed after lower build ID, so limiting concurrent jobs would work by accident and probably temporarily.

Consider when we bump the version of JSS, then update PKI to dep on the new JSS (and, some feature in it)? If we submit those builds to a repo with an existing tree, it still won't build because PKI will dep on nvr+1, which is building in parallel to JSS.

Isn't this re-bootstrap action, then?

@edewata proposed making the RFE be "add support for parsing the dependencies of the submitted jobs and building them in order based on that"

This is IMO almost no-go goal, which would definitely lead to some ugly heuristic in the best case.. consider dynamic build requires, arch-specific buildrequires, etc.

Well, we have now - for the purpose of modular builds - some logic implemented for ordering the builds (we can define that build A must be done after build B). We could implement some chain build support for your case, so you could copr build --build-after <buildID>. Would that help?

@praiskup I think copr build --build-after <buildID> would be the best solution for us. If that's possible to implement with --no-wait so that it gets submitted and doesn't block the script execution, I'd love it.

Is this something that might be feasible for a drive-by contributor to try implementing and test out or is the setup involved too high?

Thanks!

I just refreshed my memory; we don't have support for making build dependent on other build, but rather batches of builds which can depend on each other.

So using --build-after would mean (if they don't yet exit) allocating two chained batches, and assigning those builds into the batches. At the same time, it would be convenient to have --build-along <build_id>.

Not terribly difficult -- you can give it a try (you just should let us know before you start) -- but I'd like to hear opinions of teammates first. @jkadlcik, @msuchy, @dturecek ?

The proposal here would be to provide new command-line/web-UI options for build,
named --build-after <ID> and --build-with <ID>. The main reason to work with
build IDs is to not expose the batches to the end-user (they'd have to create them,
name them somehow, etc.).

The semantics of --build-after BUILD_ID would be:
1. if the BUILD_ID doesn't belong to any batch, create one and place it there
2. create new batch for the freshly created build, and place the new build there

The semantics of --build-with BUILD_ID would be:
1. if the BUILD_ID doesn't belong to any batch, create one and place it there
2. if the BUILD_ID belongs to any batch, fail if the batch finished already
3. place the new build id to the batch

As an enhancement, it would be nice if we stated in the /pending/ tab
that the build is in pending because it depends on other batch which is not
yet finished.

@praiskup, I like the idea and I think, it should be fairly simple to implement this, because AFAIK we already have batches concept working. We just need this CLI/UI parameters for that.

1. if the BUILD_ID belongs to any batch, fail if the batch finished already

When you want to build something within a batch, you don't care in which order the packages will be built. If the package is going to be submitted after every other package in the batch finished, it should build successfully. Nevertheless, I agree, that internally it doesn't make much sense - once the batch is done, it's done. I would propose to not fail the build, but rather not allow to submit it into a finished batch in the first place. What do you think? I guess that's how you meant it, right?

I guess that's how you meant it, right?

Correct, I wrongly spelled the proposal. I meant "fail" the "build- submit-attempt".

More complicated detail ... The thing is what to do with particular build failures.
If some build fails within batch, we should probably keep waiting (certain period of time)
whether user doesn't want to re-submit those failed builds. It would certainly go wrong
if we either started the depending batches after failure, or if we failed the whole
build transaction without a chance to recover.

This issue has been migrated to GitHub: https://github.com/fedora-copr/copr/issues/854