#9464 move resultsdb off fedora-31
Closed: Fixed 3 years ago by kevin. Opened 4 years ago by kevin.

Fedora 31 goes EOL soon. We need to move resultsdb01 and resultsdb01.stg off it to something else.

We could move them to f33, or el8. Should investigate what runs on them and if it's easily available in el8.


I've raised this with management a few days ago. Technically we can do the migration, but it would be nice to know to which OS we should migrate, so I'm trying to find who can give us that answer

Metadata Update from @pingou:
- Issue assigned to pingou

4 years ago

Metadata Update from @humaton:
- Issue tagged with: medium-gain, medium-trouble, ops

4 years ago

Metadata Update from @smooge:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

4 years ago

Metadata Update from @asaleh:
- Issue priority set to: Waiting on External (was: Waiting on Assignee)

4 years ago

We probably need to ask Leigh, if this is our responsibility and agree on the resultsdb owner

We probably need to ask Leigh, if this is our responsibility and agree on the resultsdb owner

This conversation has been going on for over a year at this point I think. We're trying to reduce the number of applications we maintain, so we do not want to take on resultsdb.

I have raised this again for resolution, this is outside of our control.

Maybe we would need to agree on a sort of a minimal responsibility from our side? Ee are responsible for Bodhi (and its gating feature depends on resultsdb) and we are running resultsdb on our infrastructure.

One reasonable stance could be "We are responsible for running the resultsdb service on our infrastructure to support Fedora QA, we are not responsible for development, and bug fixes." and as far as I looked into our ansible repo and at the public resultsdb source, that seems to be the current de-facto state?

This is what we are trying to get agreement on, we will run it, submit PRs, review and help where we can but we don't have the authority to merge code as this is an upstream/downstream relationship and we ultimately don't want to own that code.

And have a way/person/group to reach out to in case we encounter issues with it that we cannot solve.

Any news here? How long do we want to keep running this in a limbo state?

Any news here? How long do we want to keep running this in a limbo state?

AFAIK it's still being discussed, it's moving, just slowly.

@lgriffin do you agree?

I should have a firm direction on this next week, stay tuned

So upstream has confirmed that resultsdb runs on EL8 as well as any recent Fedora releases.

At this point while the question of long term maintenance of resultsdb still needs to be officially answered, I think we can look at deploying resultsdb to a new OS.

would EL8 be a good fit for this? We woul not playing update every 6 months on this and it will help with the other use case of it in CentOS possibly.

would EL8 be a good fit for this? We woul not playing update every 6 months on this and it will help with the other use case of it in CentOS possibly.

Possibly, we'll need to check if all the dependencies are there but from a pure
python(3) version available, it should be fine.

So, I looked at this some today.

We need 3 things for f34 on it:

python-ci-resultsdb-listener
python-zipp
python-importlib-metadata

I built the first two in f34-infra tag just fine.
The last one fails tests tho, here's a scratch build:
https://koji.fedoraproject.org/koji/taskinfo?taskID=67162754

I tried f34 because its easy to do a distro-sync and see whats needed, but I'm perfectly happy going to el8 here too if we prefer.

python-importlib-metadata

The dead.package for this one says that it is in the stdlib now, so we may not need it anymore.

ok. So what shall we do here? Adjust packages to not need that one and go to f34?
Or should we move to rhel8?

ok. So what shall we do here? Adjust packages to not need that one and go to f34?

I've checked quickly the resultsdb code and that dependency doesn't seem to come
from there (or I failed at my git grep).

Or should we move to rhel8?

Moving to RHEL8 might make the most sense to avoid the yearly rebuild but your
guess is as good as mine as to which is the best OS :/

Red Hat did run resultsdb internally on EL7 at one point so resultsdb should work on at least EL7 without issue. I don't know if they ever migrated those systems to EL8, though.

I know they did want to move resultsdb over to openshift but I don't remember if that migration has happened already for production systems. IIRC, that was waiting on a smarter way to do auth than the ip whitelist in httpd that we've always relied upon and that code landed about a month ago.

@lrossett you have the openshift versions all ready? Should we cut over from the vm's to them?

This should just need changes in proxy setup to point to openshift instead of the vm(s).

We should do this after freeze.

Metadata Update from @kevin:
- Issue tagged with: unfreeze

3 years ago

It is using fedora 34 only.. we can support more versions if we need to, but yes, we can migrate it over but I would rather test one resultsdb client app before doing it, could the issue be assigned to me?

Metadata Update from @kevin:
- Issue assigned to lrossett (was: pingou)

3 years ago

ok, so openqa auth is all sorted out.

Do we need to change/adjust fedora-ci any for this? or is that already done?

Shall we switch staging over and test with that? I'm happy to do the ansible side of things as long as others are available to debug problems with it.

Shall I go ahead with stg now?

Fedora CI doesn't talk to resultsdb directly. We send test results on a message bus and rely on ci-resultsdb-listener to ingest them to the database.

https://pagure.io/ci-resultsdb-listener

ok. We are close to getting stg working. ;)

@lrossett perhaps we should schedule a time next week to work on it?

Once we get stg working, we can roll prod out after freeze.

This is now done in prod!

Thanks to @adamwill and @lrossett !

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 years ago

Log in to comment on this ticket.

Metadata
Boards 1
ops Status: Triaged