@psss
What options do we have in TMT/FMF to describe preparation of the test environment?

We have install prepare plugin, which CI can fill in with list of rpms to install. I need to check the code more closer, but possible issues with the current implementation include:

• no support for excluding some of the packages
• need to check if all required use cases work well (reinstall, downgrade)

I requested a similar feature in Issue #29. For our use case, what we need is:

• A way to specify that none of the packages are installed by default.
• A way to install some of the packages at an arbitrary point during the test run (e.g. I should be able to do dnf install foo from the testing script and install the version of foo that triggered the test run).

I requested a similar feature in Issue #29. For our use case, what we need is:

• A way to specify that none of the packages are installed by default.

I believe this is doable. We can disable installation of packages completely from CI via a knob. CI will just prepare a repository with the tested artifact (koji build) and add it with high priority.

• A way to install some of the packages at an arbitrary point during the test run (e.g. I should be able to do dnf install foo from the testing script and install the version of foo that triggered the test run).

Sure, during the test you should be able to easily install packages which you want and thanks to the prioritized repository the ones from the package which triggered the run will be executed.

There is just one caveat here, you will need to make sure to follow correct installation procedure in case the package version is the same (in case of scratch builds this can happen), or possibly a downgrade. But that is something we need to warn the users, if they take over the artifact installation, they are on their own.

This is blocked on tmt supporting this. I will link a github PR once I have it.

Stories are being defined in https://github.com/psss/tmt/pull/479/ for TMT support

This will take at least January to get it fixed ...

Just wanted to add a comment here that this is important for glibc also which has conflicting noarch header packages. You will never need to install them at the same time. Fixing this in the toolchain would be very difficult, short of reverting the current noarch header split.

It would be great to have this fixed. I really want to test core runtimes in Fedora to make sure nothing is getting broken by our weekly package updates.

This will be our focus in January, most of the folks are already on PTO. Will update this ticket with progress. I hope that we could have at least a workaround available in the first weeks to get you unblocked.

Miro, any update on this?

Hello, any update on this? My fedora CI pull request can't be merged because of this issue.

Working hard to get this available to you. currently the idea is that you will be able to list the packages which should be installed in the plan via the prepare step in tmt in the first version.

/plan1:
  prepare:
    how: install
      package:
        - glibc
        - glibc-tests
/plan1:
  prepare:
    how: install
      package:
        - glibc-compat
        - glibc-foo

@skolosov @mcermak guys can you comment on above proposal, if it will be at least usable until we have also some kind of excluding around?

Not 100% sure, but I lean towards some sort of exclusion syntax. That is a way to specify packages that should be excluded. And install the rest of the build.

... and if some basic wildcards can be used, great! Full support for regexps would be amazing.

Miro, thanks a bunch for looking into this!

Exclusion syntax will require changes in #tmt I believe ... but I need to check ...

Nah, it is almost 2 weeks, again, hoping to finally get this done this week. Sorry, still something what has higher priority :(

Miro, any update?

Yep, I have the feature implemented mostly. Will do my best to deploy it this week for early testing.

Miro, any update on this?

Sorry we had to roll back the deployment, as it does not work well :( I will let you know once it is finally deployed and actually works, sorry for the delay :(

Miro, any update on this?

The new exclude option has been added to the prepare step specification and is now supported in tmt itself (released today in tmt-1.4). Testing Farm could use this information for excluding packages during the artifact installation as well.

Thanks a bunch for implementing this, much appreciated. Please, let us know when this gets deployed, i.e. when the production CI workers will start using this by default. Thanks!

@mvadkert Hey, Miro, any progress on Deploying this?

@mvadkert Hey, I know you told me yesterday about the reimplementation of this. But let's be transparent. Something is happening here, we're not yet there..

Me and Sergey are still very much interested in this. There are at least two eager consumers of your work. Keep up and go ahead! ;)

yeah, will stop trying to play the dead :bug: sorry, will do my best to unblock you

Miro, any update on this?

@mvadkert any update here please?

Bump; are there any news or possibly something we could help with?

@mrc0mmand we can talk about some help, would be nice, the first try to get this fixed failed, and I am having trouble to find some time to get it implemented correctly...

@mrc0mmand we can talk about some help, would be nice, the first try to get this fixed failed, and I am having trouble to find some time to get it implemented correctly...

I guess we can work something out, feel free to ping me on IRC once you have some spare time (RH or LiberaChat).

Just a note, I hit this trying to add CI for qemu. qemu generates two mutually exclusive subpackages, qemu-user-binfmt and qemu-user-static, which both want to install binfmt handlers on the host for the same architectures.

@mvadkert Hello! Any update here?

Bump; any news?

So with Zuul I have a working version for excluding packages, testing now with Fedora CI:

https://src.fedoraproject.org/rpms/curl/pull-request/9

I believe we could release this finally tmrw. Sorry for the delay.

Ooooooh, Thanks, Miro! We'll give it a try next week. Sounds amazing.

nah, did not make it, but Monday it is! will give :green_apple: light here once it is ready for you

@mcermak @skolosov @mrc0mmand @crobinso @pnemade @psss @codonell sorry this took ages, was quite fun to implement it properly. For Fedora CI jobs this now live:

Here is a proof:

https://src.fedoraproject.org/rpms/curl/pull-request/9

Example tests using exclude:

https://src.fedoraproject.org/rpms/curl/pull-request/9#_2__6

TODO:

• fix for Zuul based jobs
• fix for Fedora Copr Builds (Packit)

if somebody would be able to test this also would be nice

Marking as resolved, works in RHEL nicely:

https://gitlab.com/redhat/centos-stream/rpms/maven/-/merge_requests/7/diffs

Marking as resolved, works in RHEL nicely:

https://gitlab.com/redhat/centos-stream/rpms/maven/-/merge_requests/7/diffs

Does this apply to jobs started via Packit as well?

@mrc0mmand unfortunately not, Zuul CI and Packit still needs more <3 Filing and downstream issue for you so you can follow.

We have a task this sprint to explicitly specify which packages are meant to be installed in the testing farm session: https://github.com/packit/packit-service/issues/642

should there be a configuration in packit so you can define the list yourself?

@ttomecek hmm, I guess that issue does not make sense anymore. The exclusion of packages is directly configured in tmt plan ...