Learn more about these different git repos.
Other Git URLs
Hi, I'm moving this from https://pagure.io/fedora-infrastructure/issue/11888
Currently the %limit_build and %constraint_build macros don't work properly. The goal of these macros is to limit build parallelism based on the amount of RAM available on the builder. But they don't know about koji "weights" and fail if multiple builds can be scheduled at the same time. For example:
The problem is the macros don't know about how many other concurrent builds may be scheduled. E.g. If %limit_build knew that two builds may be scheduled at the same time, it would simply half the level of parallelism and there would be no problem.
The easiest solution is to ensure no more than one build gets scheduled at a time if a package is using %limit_build or %constraint_build. Since RPM package builds generally parallelize extremely well, this possibly won't actually slow things down too much? (Builds would take longer to start, but then complete faster.) We might need to manually attach "heavy" packages to particular build channels to achieve this. Fortunately, there are probably not too many such packages.
I'm not sure why anyone thought this could be effectively implemented without Koji's involvement.
Strictly limiting builds to one per host is simply a non-starter, though you could approximate that by setting host capacities quite low, or lowering the kojid maxjobs setting. I wouldn't recommend actually doing this in general.
This is a hard problem to solve, and I'm not sure that rpm macros are the best way to specify this information.
Metadata Update from @mikem: - Custom field Size adjusted to None
Background info on the macros -- https://fedoraproject.org/wiki/Changes/MemoryConstraintsMacros
Guess: everybody incorrectly assumed koji would never schedule multiple jobs on the same builder at once? I agree the fundamental problem is the macros don't know about the koji configuration.
The goal of these macros is to limit build parallelism based on the amount of RAM available on the builder.
I've been in need of using %constrain_build lately. Looking at the macro, it determines the total amount of memory in the system, not the total amount available. That is, the next build using that macro will see exactly the same amount of total memory, no matter how much of that is currently being used by another build on the same builder.
Unless I'm misunderstanding something, delaying the start of builds with the way the macro works, won't make any difference. Of course, the issue of overcommitting you describe, is still something worth taking into account.
Mock doesn't configure memory constraints when invoking systemd-nspawn, so we always see the total system RAM. In theory, it's possible to have Mock lock down how much CPU and RAM is allocated when using nspawn to host the build environment.
Log in to comment on this ticket.