@lruzicka said he wanted to work on this, so assigning to him.

Thanks, I am taking it for now and will investigate it.

So, I have been working on this for a while. My achievements so far:
I know where the data resides.
I reorganized them to be environment agnostic.
I studied and implemented a simple Jinja template to see how things work.
I had to create output to support the Jinja template.

I still have to connect the data with the Jinja properly and see if I can get the correct output.

I have been able to generate a solid html output (see attachment) using

• Jinja
• CSS
• a bit of Java Script

To do:

• fix the JSON output accordingly
• fix the output location based on user selection
• clean the code and provide comments where needed

I generally try like hell to avoid including javascript in anything I maintain, but if it's sufficiently useful we can look at it. your layout does look a bit nicer for sure.

I don't know if I understand the output correctly, but it seems you know show a single field, regardless of how many test runs occurred on that compose? I'd like to keep the previous style, where you see the amount of tests performed:
⠄= 0-2 results; ⠆= 3-4 results; ⠇= 5+ results

I find it very useful.

Also, I often use column sorting (e.g. by Last In, or multisort [1] Release Level -> Last In), let's please keep that feature :-)

[1] From the original agenda: TIP! Sort multiple columns simultaneously by holding down the shift key and clicking a second, third or even fourth column header!

Currently, the tests are presented with a heated map, one symbol for a compose, split into environments, they HTML is backed with Bootstrap. I talked to @kparal and he has some more suggestions for enhancements that I would like to work on.

Right, I'll try to remember it all :-)

I find it very useful to make the environment represented basically in the very same way as it looks in our wiki matrix, i.e. testcases are rows, environments are columns. In each cell, instead of a single result, there's a history of results. This makes the layout more compact (similar to the old system) and easier to scan with your eyes. If the page gets too wide, perhaps we'll need to limit the amount of results displayed in each cell (e.g. last 15 composes instead of all of them). Or we can wrap the result cell and the dots would then flow in several lines (it would be less compact, but perhaps still readable enough).

Each result dot should be a hyperlink to the corresponding result matrix, so that we can look at the original page easily and see what bugs were reported by whom, etc. Or it can be some popup with details, and then a hyperlink.

I love distinguishing bot results from human results, let's keep that!

The heatmap could be perhaps less detailed (maybe just 3 different colors), to make it easier to read and avoid making "lots of results" dots too dark. Or we need to change the palette, and go from a pale color to the most shining one.

I don't find the coverage column useful, I'd remove it (more horizontal space for us). It's easy to see in the row of dots anyway.

It would be nice to highlight the testcases which need attention. The actual algorithm will surely need some fine tuning, but if we add some background highlight for rows where the release level is not optional (i.e. mandatory) and no result has been posted in the last e.g. 10 composes, that might draw our attention well to the required tasks.

I wonder if we could make the "Last In" column sort properly even for cells without a date, but containing e.g. "Beta". Perhaps it would ignore the actual contents of the column, and sort it according to the Results column (the number of grey dots at the end of the graph)? Or drop the Last In column completely, and just make the Results column sort this way? But I expect that to be quite difficult, I'm just imagining the best possible output :-)

Have I forgotten anything that we talked about in person, Lukas? Not sure.

The majority of requests are already implemented, I am just finalizing the PR right now and brushing up on the last cosmetic things.

This is now fixed by https://pagure.io/fedora-qa/relval/pull-request/31 . Thanks, Lukas!

Hopefully this will make testing easier for teams like KDE during F42 release validation.