#12112 The process to update the OpenH264 repos is broken
Opened 7 months ago by abompard. Modified 9 days ago

The process to update the Cisco OpenH264 repos is no longer valid since MirrorManager's move to Openshift. We can't copy the files on mm-backend01 and run a script there anymore.

MirrorManager could automatically scan the repo, but it must be made available via a NFS mount.
Could we do that?
MirrorManager is currently mounting the primary mirror export and the archive export, but I don't think those will do since we want to keep the codecs private.


Metadata Update from @zlopez:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: medium-gain, medium-trouble, ops

7 months ago

Yeah, so possible options:

  1. Could make a small seperate nfs volume for this so it can be updated by releng and used in openshift for mirrormanager? yes, we want to be very carefull to never expose the packages out to the world.

  2. Could make some kind of volume in openshift and figure out some way to sync data to it?

I suppose here we only need the metadata, so it could be small, but it does update somewhat regularly...

Another option (although perhaps more complex):

  1. Openshift has a 's3 compatible' storage built in. We use it now to pull websites to proxies. Perhaps we could set it up so we could push to a volume the contents...

We likely need this very soon due to releng 12275

Related issue: https://pagure.io/releng/issue/12275

I’ve received confirmation that the request has been received by the current maintainers of the publishing process and should be completed in the next couple of days.

Is there a plan on how to proceed when that happens?

@kevin @abompard

I just received confirmation that the binaries are now published and available, and indeed they are.

$ curl -I http://ciscobinary.openh264.org/openh264-2.4.1-2.fc41.x86_64.rpm
HTTP/1.1 200 OK

Is there some temporary workaround that we could do here?

@kevin @abompard

I'd defer to @abompard here. If you want me to make a nfs volume I can...

Yeah, please do that. I think an NFS volume would be easier to share between Openshift and the rest of our infra. Thanks!

ok. I have created a 'openshift_prod_codecs' volume.

It's mounted on /mnt/ on sundries01 right now, and I have copied the existing /srv/web/codecs.fedoraproject.org/ content to it.

I think we should probibly just mount it at /srv/web/codecs.fedoraproject.org, but that will take a freeze break right now, or we could wait until after freeze.

We still need to create a pv in openshift and setup mm to mount it. @abompard can you do that part?

We also need to copy the new content to it (the latest we sent to cisco). @patrikp can you do that part?

Moving forward we also need to setup something to compose these thats not ODCS (since we will be retiring it very soon).

Thanks, I'll deal with the openshift part. It looks like this mount isn't available from staging, am I correct? If so, do we have an equivalent share for staging or should I just ignore it there?

Do you think this change would require a FBR in prod? I'd think so.

I didn't make a staging one... since we never really made staging composes of this. I'd say just ignore on staging...

If we do want to add staging we could, but it would mean we would need to do builds there of it, then composes of the repo, then have some way to test it, which I guess would require asking cisco if they could do another staging repo and send them rpms for it? But that seems like a big lift. ;(

Sure, now that the beta freeze is over I'll just update prod :-)

OK, the Openshift part is done!

ok. I mounted the volume on /srv/web/codecs.fedoraproject.org on sundries01 now.

So, I think now we are set to have @patrikp sync the new repodata to /srv/web/codecs.fedoraproject.org on sundries01 and mm should pick up the new repodata and start serving the new rpms on cisco's side.

Did I miss anything here?

Metadata Update from @patrikp:
- Issue assigned to patrikp

6 months ago

I'm getting a "read-only filesystem" error when trying to sync the repodata.

Could this perhaps be related to the btrfs bug that you mentioned in the weekly releng meeting the other day, @kevin?

Nope. :) It was that I didn't set it to be read-writable. ;(

Should be fixed now, try again?

It works, should be synced now. :thumbsup:

@abompard can you see if mirrormanager updated? I'm getting reports that it's not showing new version in rawhide...

As I replied on https://pagure.io/releng/issue/12275#comment-937076, I don't see a rawhide directory in the codecs space. So the Mirrormanager does not recognize a rawhide repo and does not add it to the database.

OK, it seems that everything works as expected and the releng ticket [1] is now closed.

We have another problem and that is that odcs (used to generate the OpenH264 composes) was recently retired. [2][3]

This ticket is about the process of updating the repos, so I suppose it can be closed and another one should be opened to talk about how to generate the composes in the future?

We already have a request for sending openh264-2.4.1-1.el10_0 to Cisco. [4]

[1] https://pagure.io/releng/issue/12275
[2] https://pagure.io/fedora-infrastructure/issue/12192
[3] https://pagure.io/fedora-infra/ansible/pull-request/2266
[4] https://pagure.io/releng/issue/12385

CC: @kevin @jnsamyak

OK, it seems that everything works as expected and the releng ticket [1] is now closed.

Excellent!

We have another problem and that is that odcs (used to generate the OpenH264 composes) was recently retired. [2][3]

This ticket is about the process of updating the repos, so I suppose it can be closed and another one should be opened to talk about how to generate the composes in the future?

Well, sure... but we could just keep this ticket for that.

My first thought is that we could make a small pungi.conf and use pungi on a internal releng machine to generate these repos.
We don't want to use dist-repos (because they would be public), we don't want to just make a script to make them because there's things like multilib and such that would be tricky.

I am not sure what the Fedora Compose is, but I can not watch x264 encoded videos without new 2.5.0 version of OpenH264 released to F41, so can anybody explain in a layman terms what needs to be done, and why it is impossible to do?

@abitrolly in the context of fedora releng compose is a process that takes the RPMs and creates repositories out of them.

The thing is that we used ODCS project previously to create the repos. That tool is now retired.

Another thing is that we do not have legal rights to distribute the content so this process needs to happen "privately".

So, we talked about this in a releng meeting recently and decided that koji dist-repos might work here.

@patrikp did that work for this? can we send to cisco?

Hi, our rpm-install test in dist-git is failing because of the gpg key of the CISCO repository. Is it fine to ignore this test until the GPG keys are resolved? I think it should be as we are working fine even with the stub package which is used everywhere else.

https://fedora.softwarefactory-project.io/zuul/build/52ea8f0d94654ae3b664194d8cc8b9ec

So, we talked about this in a releng meeting recently and decided that koji dist-repos might work here.

@patrikp did that work for this? can we send to cisco?

Yes, I think dist-repos is the way to go. However, OpenH264 version 2.6.0 came out in the meanwhile (a couple of months ago) and I don't see any builds for this version.
As we discussed in the releng weekly meeting, perhaps it would be best to get the 2.6.0 builds and send them all at once, so I asked the maintainer about it here and I am currently waiting for a reply:
https://pagure.io/releng/issue/12385#comment-956834

Update:
I have good news, I think. We still have to wait to see if it works but I went through the entire process. Generated the RPMs through koji dist-repo, sent the tarball to Cisco, they updated their CDN, and I just synced the repodata to sundries. If nothing went wrong then we can close this ticket, leaving it open for now...

Relevant ticket:
https://pagure.io/releng/issue/12617

So, there was some mirrormanager issues.

  • The rawhide 'alias' for this was pointing to f42. I changed it to point to f43. (The web UI gave a 500 trying to edit it, so I just changed it in the db).

  • There is a slight change in the repo setup here... the old ones had a /os/ directory in there before the repodata dir. The new dist-repo one doesn't. ;(
    I tried to fix this in mirrormanager, but it doesn't seem to have worked. ;(

@adrian or @abompard could you take a look?

Hmm it looks like you did was is necessary, no?

$ curl "https://mirrors.fedoraproject.org/metalink?repo=fedora-cisco-openh264-43&arch=x86_64"
<?xml version="1.0" encoding="utf-8"?>
<metalink version="3.0" xmlns="http://www.metalinker.org/" type="dynamic" pubdate="Thu, 13 Mar 2025 12:30:00 GMT" generator="mirrormanager" xmlns:mm0="http://fedorahosted.org/mirrormanager">
 <files>
  <file name="repomd.xml">
   <mm0:timestamp>1741171556</mm0:timestamp>
   <size>3082</size>
   <verification>
    <hash type="md5">7d0f9f3406e223710a2de9f73380aeb0</hash>
    <hash type="sha1">efbd91d67445ae1331b7992371df99ac1a34faef</hash>
    <hash type="sha256">5f47b7a23a7a5366ea8a89c9e2ce3dd9cfe32834daf7f4d193b2ef8297cece78</hash>
    <hash type="sha512">ee95925e58f3e99dd1aecab63dc7ec8008c28a35d14d3acafdf5ae41506254be4fc29732c294c709eb15dfbd3fd09ce3ad23614f5446dfda24ca5d422ddebc0e</hash>
   </verification>
   <resources maxconnections="1">
    <url protocol="https" type="https" location="US" preference="100">https://codecs.fedoraproject.org/openh264/43/x86_64/repodata/repomd.xml</url>
   </resources>
  </file>
 </files>
</metalink>

Or maybe I'm missing something? Is something else wrong?

I think @adrian fixed it. :) From matrix in the #releng:fedoraproject.org room:

nirik: I am looking into the codecs f43 problems now
nirik: it should be fixed. I think the problem was that there was content with the old layout (with /os/) there and once MirrorManager detects a repository it will not move it, so the repositories had to be deleted from the database and then be redected

The old layout was my fault, I copied the old 'empty' repo we have for updates... and that has the pungi style os/ dirs. ;(

Anyhow, this is seeming fixed for rawhide (although there's dep issues which are being worked on before the update will actually install).

@patrikp Shall we close this one now? Or ?

Log in to comment on this ticket.

Metadata
Boards 1
ops Status: Backlog