It would be useful to have only the packages listed in the required_packages: tag installed into the testing environment. The current process of installing all sub-packages of the SRPM being tested can hide implicit dependency bugs that may be present.
For example, if we are testing sub-package B and sub-package B should have an explicit dependency on sub-package A, but does not, then this bug will never be caught by CI, since A and B will always be installed together in the testing environment.
In my opinion, this kind of dependency checking is an important use case for CI, so it would be nice to have a little more control over the testing environment setup to make sure we can catch these kinds of issues.
I thought about this as well and was surprised by this implicit behavior.
Dependency problems should be caught by different set of tests really (installatin and rpm dependency sanity with rpmdeplint). Maybe we could open seaprate issues to have those enabled for Fedora.
Can those tests catch dependency issues that only appear when you run a program? Or when you run a program with certain command line arguments? The kind of test case I'm thinking about is like this one: https://src.fedoraproject.org/rpms/clang/blob/f29/f/tests/testspocl.yml
@tstellar of course not :) ok so what I am hearing is that you want to control packages which are installed from the tested artifact. Sane default is all packages, you want to be able to override that. Can we phrase this way your requirement pls?
Sure, I will rephrase my request:
I would like the option to be able to control which sub-packages from the tested artifact are installed into the test environment before any of the tests are run.
So for some time now, we can disable certain packages:
https://mailman-int.corp.redhat.com/archives/rhel-devel/2021-August/msg00352.html
Few packages use that, it should help in most of the cases. Specifying a specific list is not yet possible :(
ok, there is naother issue related to bodhi. Not all packages from an bodhi update should be installed ...