@msrb could we test suite error not qualify as infra error pls?

<testcase name="selinux/policycoreutils/sctp_test" result="error">
<properties>
<property name="baseosci.arch" value="x86_64"/>
<property name="baseosci.connectable_host" value="172.31.23.27"/>
<property name="baseosci.distro" value="Fedora-Rawhide"/>
<property name="baseosci.status" value="Complete"/>
<property name="baseosci.testcase.source.url" value=""/>
<property name="baseosci.variant" value=""/>
</properties>

Adds an error here:
https://github.com/fedora-ci/jenkins-pipeline-library-agent-image/blob/master/scripts/tfxunit2junit.py#L211

If you want to fix the error, it is in STI:

from http://artifacts.dev.testing-farm.io/9ca462f4-4b4f-47f2-ae8d-b9159f00cbbe/work-tests.ymldSPb8L/tests-tKWCMm/results.yml

- test: selinux/policycoreutils/sctp_test
  result: error
  logs:
  - ERROR-selinux-policycoreutils-sctp_test.log
  - ERROR-selinux-policycoreutils-sctp_test-err.log

@mvadkert not sure if I am following what you're saying here. It's a testcase marked as an error in xunit. So that's why it is shown as an infra error in Jenkins.

Overall result is "failed" (taken from TF API response): https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/dist-git-pipeline/job/master/217/

@bgoncalv could you please check why the sctp test errored here? http://artifacts.dev.testing-farm.io/9ca462f4-4b4f-47f2-ae8d-b9159f00cbbe/work-tests.ymldSPb8L/

You're more fluent with STI than I am :)

@mvadkert not sure if I am following what you're saying here. It's a testcase marked as an error in xunit. So that's why it is shown as an infra error in Jenkins.

yep, but the error comes from STI, I just consume results.yml which it generates

@bgoncalv results.yml is in http://artifacts.dev.testing-farm.io/9ca462f4-4b4f-47f2-ae8d-b9159f00cbbe/work-tests.ymldSPb8L/tests-tKWCMm/results.yml

In this case we should send the same topic as test failure and not as infra issue. The beakerlib role distinguish between test failure and test error, but this difference could be show when displaying the result, but the message topic should be the same as for test failure.
When executing STI playbook, we should report as infra failure if the playbook exited with value different from 0 or didn't create test.log.

@bgoncalv thanks for the clarification. We send complete/failed messages in this case. It's the STI infra issue that's confusing. In Jenkins, we show what results.yml/xunit says, and that means telling people that this one test infra-failed (overall result is still a standard failure).

The question is: what happened in STI in this case? Why the infra failure?