This ticket comes out of some idle fiddling I did today, and bouncing some ideas off @mvadkert . We've both been thinking about the current workflow with results and messages. Today, it looks like this:

1. Fedora CI -> message bus -> ci-resultsdb-listener (process/flatten) -> resultsdb (-> message bus)
2. openQA --> message bus
   -> resultsdb (different format) (-> message bus)

(that is, openQA publishes to resultsdb directly, ci-resultsdb-listener does not know how to parse its messages at present).

There's a clear drawback that's common to both flows: what's stored in resultsdb is significantly different than what's published on the message bus. Both Fedora CI and openQA are (more or less...) publishing messages in ci-messages format. But what they're publishing to resultsdb is significantly different. For Fedora CI, it's a kinda processed/flattened version of the CI message, produced by ci-resultsdb-listener (in Fedora, I believe internally something else does this). For openQA, it's a pretty different format produced by my own resultsdb_conventions, which is a kinda overengineered attempt to achieve the same 'consistency' goals as the ci-messages spec. So we have three representations of similar data.

Miro has an idea to just have the result systems report to resultsdb directly and skip message publication. Of course, there would still be a message - the one published by resultsdb. Anything that needed a message to consume could consume that message.

I have an idea to have openQA adopt the ci-resultsdb-listener system, improving ci-resultsdb-listener to handle openQA messages (and, by the by, also handle Fedora CI pull-request messages, because it doesn't handle those, so those results never make it to resultsdb...)

This afternoon I've been playing with something relevant to both ideas. Neither openQA nor Fedora CI currently uses this, but it is actually possible to submit structured data as the values in the extradata dict sent to resultsdb, not just strings. See this test script, which dumps the whole body of a real CI test complete message into resultsdb, then proves it can parse it back out again:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

#!/bin/python3
import ast
from resultsdb_api import ResultsDBapi, ResultsDBapiException, ResultsDBAuth
MSG = {
    "artifact": {
      "alias": "FEDORA-2023-c17d97e7e9",
      "builds": [
        {
          "nvr": "chrony-4.4-0.3.pre2.fc39"
        }
      ],
      "id": "sha256:4e5f7a80c8e2880dc1356f000adde402bc3d0b5efb9682f3fb56ac339c6e6a56",
      "release": {
        "name": "F39",
        "version": "39"
      },
      "type": "fedora-update"
    },
    "contact": {
      "docs": "https://fedoraproject.org/wiki/OpenQA",
      "email": "qa-devel@lists.fedoraproject.org",
      "irc": "#fedora-qa",
      "name": "Fedora openQA",
      "team": "Fedora QA",
      "url": "https://openqa.stg.fedoraproject.org"
    },
    "generated_at": "2023-06-21T15:29:05Z",
    "pipeline": {
      "id": "openqa.Update-FEDORA-2023-c17d97e7e9.rpmostree_overlay.64bit.updates-silverblue-dvd_ostree-iso.x86_64",
      "name": "openqa.Update-FEDORA-2023-c17d97e7e9.rpmostree_overlay.64bit.updates-silverblue-dvd_ostree-iso.x86_64"
    },
    "run": {
      "id": 2946911,
      "log": "https://openqa.stg.fedoraproject.org/tests/2946911/file/autoinst-log.txt",
      "url": "https://openqa.stg.fedoraproject.org/tests/2946911"
    },
    "system": [
      {
        "architecture": "x86_64",
        "os": "fedora-39",
        "provider": "openqa",
        "variant": "Silverblue"
      }
    ],
    "test": {
      "category": "validation",
      "namespace": "update",
      "result": "passed",
      "type": "rpmostree_overlay 64bit updates-silverblue-dvd_ostree-iso x86_64"
    },
    "version": "0.2.1"
}
authmethod = ResultsDBAuth.basic_auth("someuser", "somepass")
rdb_instance = ResultsDBapi("https://resultsdb.stg.fedoraproject.org/api/v2.0/", request_auth=authmethod)
ret = rdb_instance.create_result(outcome="PASSED", testcase={"name": "__ADAMTESTIGNORE__", "ref_url": "https://www.google.com"}, ref_url="https://www.google.com", resultdata=MSG)
res = ret["id"]
resdata = ast.literal_eval(rdb_instance.get_result(res)["data"]["resultdata"][0])
print(resdata.keys())

You need a valid username and password, but I've run that and it works :P This is the result it creates - it looks awful in the web interface but that'd be easy to tweak. The representation is slightly cock-eyed: for some reason get_result always wraps the extradata values in lists - I've no idea why it does that - and the dict keys in the string are wrapped in single quotes, not double quotes, which is why you have to use ast.literal_eval, you can't use json.loads because single quotes aren't valid JSON. But those are relatively minor things, the fact is it works. You can dump a fairly complex nested dict into resultsdb and get it back out again fairly easily.

So, whichever way we want to do it, we can - if we want - store something like the current ci-messages message data into resultsdb and get it out again, either directly or via ci-resultsdb-listener. The flattening that ci-resultsdb-listener does on the data at the moment isn't really necessary. I don't know if we ultimately want to do this, but it's an option.

Anyway, that's as far as I got today poking around at things :) Of course this is complicated by considering how things work internally in RH, but we already have some divergence there (e.g. I don't think we use ci-resultsdb-listener internally any more). It'd be great if we can agree a plan to reduce the amount of mucking around we do with the representations and simplify the process, while making openQA and Fedora CI resultsdb data more similar. This would also help with onboading more systems: I know the Fedora CoreOS folks are interested in having their own CI system report into resultsdb (because I've been talking about it with them).

Anyone have thoughts?