#11711 Update compose hosts to get latest pungi release (4.6.0)
Opened 4 months ago by siosm. Modified 3 months ago

Describe what you would like us to do:


Update the packages on the hosts used for the compose to get the latest pungi release (4.6.0).

This is needed for the ostree native container work. See:
- https://pagure.io/pungi-fedora/pull-request/1227
- https://fedoraproject.org/wiki/Changes/OstreeNativeContainerStable
- https://pagure.io/releng/issue/10399

We also need the ELN compose hosts to be updated as well for: https://pagure.io/pungi-fedora/pull-request/1226

When do you need this to be done by? (YYYY/MM/DD)


As soon as possible, preferably well before Fedora 40 beta so that we can test the new images early.


Metadata Update from @kevin:
- Issue assigned to kevin
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: low-trouble, medium-gain, ops

4 months ago

Rawhide/branched composers are easy. Done.

ELN is going to be more involved. Will talk to folks and see what we can do...

@sgallagh What do you want to do here about new pungi?

I guess we could manually rebuild it and update it... but it would sure be nice if we could be on a supported release there. ;(

Is there something different about the ELN hosts?

Rawhide/branched composers are easy. Done.

Hmm well, I guess it'd make sense to do this for rawhide too.

Is there something different about the ELN hosts?

Yes. eln is using odcs to do it's composes. It's the very last/only thing doing so now. The backend machine it's using is a f36 vm. ;(

I would really prefer to move them off odcs, but we will figure something out.

We got a fix for https://pagure.io/releng/failed-composes/issue/5814#comment-892007 so we'll need another update when this lands in rawhide.

https://bodhi.fedoraproject.org/updates/FEDORA-2024-00670dd964 landed in rawhide. Can you update the rawhide/branched hosts again? Thanks!

Is there something different about the ELN hosts?

Yes. eln is using odcs to do it's composes. It's the very last/only thing doing so now. The backend machine it's using is a f36 vm. ;(

I would really prefer to move them off odcs, but we will figure something out.

I was really hoping we could get this to limp along for a couple more weeks and deal with it once CentOS Stream 10 diverges and ELN starts tracking EL 11, but it's looking like that might not be feasible.

What is it going to take to migrate away from ODCS? Should I copy https://pagure.io/pungi-fedora/blob/main/f/nightly.sh into the ELN branch and try to modify it? How do I test that?

https://bodhi.fedoraproject.org/updates/FEDORA-2024-00670dd964 landed in rawhide. Can you update the rawhide/branched hosts again? Thanks!

We need the f39 one, but I know what you meant. ;) Done.

I was really hoping we could get this to limp along for a couple more weeks and deal with it once CentOS Stream 10 diverges and ELN starts tracking EL 11, but it's looking like that might not be feasible.

I have the above pungi built for f36 ready. I don't know if it will work, but it built fine. If you like I can update it and you can see if that gets you a bit further?

What is it going to take to migrate away from ODCS? Should I copy https://pagure.io/pungi-fedora/blob/main/f/nightly.sh into the ELN branch and try to modify it? How do I test that?

I think off the top of my head:

  • I create a compose-eln01 vm (running... f39?)
  • you copy the nightly.sh to eln branch and modify it.
  • You can try it on compose-eln01 until composes under /mnt/koji/compose/eln/ finish the way you like.
  • We then need to sync that somewhere (/pub/fedora/alt/eln ?) and tell consumers to look for the content there instead of the odcs directory.
  • We wait until they are switched over and retire odcs.

If that sounds reasonable, I can make a new ticket with that info and start in on it ?

What is it going to take to migrate away from ODCS? Should I copy https://pagure.io/pungi-fedora/blob/main/f/nightly.sh into the ELN branch and try to modify it? How do I test that?

I think off the top of my head:

  • I create a compose-eln01 vm (running... f39?)
  • you copy the nightly.sh to eln branch and modify it.
  • You can try it on compose-eln01 until composes under /mnt/koji/compose/eln/ finish the way you like.
  • We then need to sync that somewhere (/pub/fedora/alt/eln ?) and tell consumers to look for the content there instead of the odcs directory.
  • We wait until they are switched over and retire odcs.

If that sounds reasonable, I can make a new ticket with that info and start in on it ?

Thanks, I'll try to carve out some time next week to work on the nightly.sh modifications.

Is there any hope that we could arrange a redirect from https://odcs.fedoraproject.org/composes/production/latest-Fedora-ELN/ to the new location, or should we look at getting it formally added to the mirror lists? We had originally tried to avoid doing that so as to limit the load, but I assume mirrors can pick and choose which releases to support?

Did you want me to try and update pungi on odcs-backend-releng01?

Yeah, we can do a redirect most likely. If we put it in /pub/fedora/alt/ it should automatically get mirrored by people who mirror alt. We could also make a new /pub/fedora/eln and make it it's own mirrormanager category I suppose.

Did you want me to try and update pungi on odcs-backend-releng01?

If we can do that to keep things running (and support the bootc work) for the next couple weeks, that would probably be the safest approach.

Yeah, we can do a redirect most likely. If we put it in /pub/fedora/alt/ it should automatically get mirrored by people who mirror alt. We could also make a new /pub/fedora/eln and make it it's own mirrormanager category I suppose.

I'd slightly prefer that; we already have one unofficial mirror (Facebook) out there; it would be best if we could just get that made consistent with the rest of the releases.

ok. I updated pungi and productmd (which newer pungi needed a newer version of). Can you see if that works?

ok, we can figure it out.

Error while generating compose: Command '['pungi-koji', '--config=/tmp/tmpvwtuaz0a/pungi.json', '--no-latest-link', '--compose-dir=https://odcs.fedoraproject.org/composes/odcs-29327', '--no-latest-link', '--production', '--label=Alpha-0.1705695155', '--koji-event', '124171316', '--old-composes', 'https://odcs.fedoraproject.org/composes/production']' returned non-zero value 1, stdout log: https://odcs.fedoraproject.org/composes/odcs-29327/pungi-stdout.log, stderr log: https://odcs.fedoraproject.org/composes/odcs-29327/pungi-stderr.log
Compose run failed: HTTPSConnectionPool(host='koji.fedoraproject.org', port=443): Pool is closed.

Hey folks, any update on this update (pun intended) :smile:

I'm not sure. @sgallagh ? I think the new pungi got reverted back.

Are we now going to try and move to a compose-eln01 ? Or is that post branching plan?

I could I suppose also try saving off the curent vm and upgrading a copy to f39 and see if things work there? But moving to compose-eln01 is much better for me. ;)

Given that we're forking EL10 on Tuesday, I'd much prefer if we just held off until then.

Login to comment on this ticket.

Metadata
Boards 1
ops Status: Backlog