#395 Aborted tests should leave link to failed run logs
Closed by mvadkert. Opened by pemensik.

I made error in my tmt plan editing by commit 930329804. But in following update it just reported test as missing without any useful details.

Link to output like console would help a lot, because it includes clear indication what exactly is wrong.

But finding such result from the update is difficult for me as mere developer. I think if the results were not simple success or failure, it should leave a comment inside update with a link to build. Whether just clickable via test id or just comment somewhere, it does not matter. But just tests failed message is not useful. With help of @mvadkert it were fixed, but I think the error reporting should be improved.

Additional problem were missing restart tests button in the update, even though I were the one who created it.


The test run were fixed by commit e853970bcf83

I made error in my tmt plan editing by commit 930329804. But in following update it just reported test as missing without any useful details.

That was probably just a delay. The console link shows the following:

[Pipeline] sh
+ tmt plan ls --filter enabled:true
    warn: /plans/all: - Additional properties are not allowed ('component' was unexpected)
    warn: /plans/tier1-public: - Additional properties are not allowed ('component' was unexpected)
[Pipeline] findFiles
[Pipeline] findFiles
[Pipeline] echo
STI tests in https://src.fedoraproject.org/rpms/bind (10cb7fcdc54db43fca7f35029f62f979a5d57251): [] []
[Pipeline] findFiles
[Pipeline] echo
FMF tests in https://src.fedoraproject.org/rpms/bind (10cb7fcdc54db43fca7f35029f62f979a5d57251): [.fmf/version]
[Pipeline] deleteDir
[Pipeline] }
[Pipeline] // dir
[Pipeline] build (Building fedora-ci » dist-git-pipeline » master)
Scheduling project: fedora-ci » dist-git-pipeline » master
[Pipeline] build (Building fedora-ci » dist-git-pipeline » master)
Scheduling project: fedora-ci » dist-git-pipeline » master
[Pipeline] error

I.e. there were separate jobs spawned for each plan.

One of them succeeded:
https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/dist-git-pipeline/job/master/224029/
And one of them failed:
https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/dist-git-pipeline/job/master/224028/console

And that is what you see in Bodhi: https://bodhi.fedoraproject.org/updates/FEDORA-2023-3ac91f36d6

That is state after Miro restarted them manually, previous state is not sure where. But there were no clickable link to any result of previous job. And that is my point. Checking why the run failed should be more straightforward. Especially when result is ABORTED or UNSTABLE.

There is no trace of any previous execution for bind-9.18.12-1.fc39. Are you sure the test actually started before Miro intervened?

No, of course not. How can I be? Is there any indication given test already started? It would be nice if in-progress indication leading to the task would be somewhere in the update. It were two weeks old before Miro intervention. I had no idea when it started. That should be improved too. It should be obvious if it is still in queue, started something and when it finished.

It may be simple for you, but us developers have no good indication whether we should still wait.

Trust me, it is not simple for anybody. There are some deeply rooted problems that stem from the overall gating design. And design problems are the most expensive problems to fix...

Another problem that wasn't mentioned is that none of the tests in your update are marked as required (prefixed with an asterisk) in Bodhi:
https://bodhi.fedoraproject.org/updates/FEDORA-2023-3ac91f36d6

On the other hand, required tests are properly marked here:
https://bodhi.fedoraproject.org/updates/FEDORA-2023-715f3b4865

The difference is that your update has more than one build in it...

I really don't know what went wrong if there are no logs (anymore). And it's hard to tell where the real problem is. Could you please comment here when you encounter the problem again?

In progress for bodhi is tracked here:

https://pagure.io/fedora-ci/general/issue/103

I am working with them, and it is being developed, but not yet there.

Feel free to :cry: there if you can, at least they will recognize more how important it is

And I agree, I said it on Slack, logs are gone, pls report issues sooner if you can next time :(

Fedora Ci is high volume, we cannot keep longer the logs here.

Bodhi status for queued / running has landed, let's close this and open a new issue if issues persist.

Metadata Update from @mvadkert:
- Issue status updated to: Closed (was: Open)

Metadata