Hi, tmt is marely a tool used to discover, develop and run tests. It does not plan anyhow to be involved in filling bugzillas. I need to understand the problem, but seems to me that we need to find, what needs to be done on CI and gating side to avoid https://bugzilla.redhat.com/show_bug.cgi?id=2021972.

@churchyard is something like our ... long coming .. koschei-like-ci test would help in catching problems like these?

I don't understand what @abitrolly really wants but no, I don't think so.

@churchyard I want to avoid filling bugs that can be caught and reported automatically by tests running in CI.

In that case, no -- this is not related at all to the koschei-like-ci test.

It could, however, be related to Koschei itself, if Koschei also run CI tests on successful builds and reported bugzillas for failures.

@abitrolly the idea behind CI and gating is to prevent breakage by blocking the change to get to the compose. So in this case, was this unavoidable by running and gating the build on some tests? Do you know?

Not sure that the compose is. But running tests for gating would prevent m2crypto from being released in Rawhide and breaking installs there. If that package still need to be released in semi broken form while it is being worked on, all test failures can be published to bugzillas, so that people who run into the these failures can find them.

I do not believe much in this "needs to be released in a semi-broken form" personally.

We really would like to avoid filling automatic bugzillas. All efforts I was part of for the years, automatic filling bugzillas was never perceived well. We should strive for gating on MR, preventing broken changes getting in.

I am afraid I cannot pursue this myself :(

Meaning somebody has to manually spend their time troubleshooting and logging problems instead of inspecting things that could have already been caught.

I am not really a Fedora packager any more, so I shouldn’t even comment on this topic here, but this seems like missing that somebody has to do bugs’ nappies changing anyway. Either it is QA people before they file the bug, or packagers have to do it to eliminate false negatives. The question is who is a more valuable resource to you, and who would get easier to offend.

However, that isn’t why I am here:

would prevent m2crypto from being released in Rawhide and breaking installs there

As the M2Crypto upstream, I am disturbed to hear that M2Crypto is so persistent source of issues that you are changing your QA because of it. What’s going on? Why I am not hearing about it more upstream (which has now moved to https://sr.ht/~mcepl/m2crypto and the preferred way for reporting issues is the mailing list https://lists.sr.ht/~mcepl/m2crypto)?

@mcepl I am not that English native to understand bugs’ nappies for QA people. Maybe try to make the point without analogy? As far as I know Fedora doesn't have resources to pay dedicated QA people to manually fill and validate bugs. That's why automation like tmt is invented.

@mcepl the specific root cause of this problem 2 years old, when M2Crypto was incompatible with Python 3.11, which is easy to catch, but still got it to Rawhide to break things. See https://src.fedoraproject.org/rpms/m2crypto/pull-request/7