#12617 Please send openh264-2.6.0 to Cisco
Opened 4 months ago by patrikp. Modified 15 days ago

It seems that we now have the 2.6.0 builds for all of the releases. [1] Thanks @wtaymans!

We also have a way of generating the RPMs and metadata through the koji dist-repos utility. The issue before was that odcs (on demand compose service), which we used for this, was retired without a plan in place.

Information about this is spread across many different tickets. In the name of clarity let me close those, put a link to this ticket in them, and put links to the closed tickets in the section below so that we can have a single place to track it.

Resources:
[1] https://koji.fedoraproject.org/koji/packageinfo?packageID=21431
[2] Please update openh264 to 2.6.0 in all supported Fedora releases: https://pagure.io/releng/issue/12585
[3] Please update openh264 to 2.6.0 in F41: https://pagure.io/releng/issue/12466
[4] Please send openh264-2.4.1-1.el10_0 to Cisco: https://pagure.io/releng/issue/12385

CC: @jnsamyak @kevin @kalev @catanzaro @wtaymans @abitrolly


Fedora/EPEL Version OpenH Version Sent to Cisco? Synced to sundries01?
F43 2.6.0 YES YES
F42 2.5.1 YES NO
F41 2.5.1 YES NO
F40 2.5.1 YES NO
EL10_1 2.5.1 YES NO
EL9 2.5.1 NO NO

I don't see how to vote for that, but you have mine.

Let's hold off on sending these to Cisco temporarily. It's not clear whether it's safe to release this update with ABI version 7 after all. We need to discuss more in https://github.com/cisco/openh264/issues/3863.

It's definitely not safe. Let's just drop these builds; we'll need to try again.

Bummer. So, for rawhide, perhaps we should just resign the existing build... or do you think it will be solved soon?

We need to rebuild with ABI version 8.

It's definitely not safe. Let's just drop these builds; we'll need to try again.

Alright. I have the rawhide tarball with RPMs ready to go, I was waiting for a double check from my teammates and was about to send it today.

The new process is now documented so when you let us know that we're good to go with the new builds we should be able to do it in a timely manner.

I appreciate everyone's patience.

There is a newer rawhide build now, openh264-2.6.0-2.fc43. Please go ahead and send that.

For all other branches, we still need to do new builds.

As a side note, I made a empty repo for rawhide/f43. Hopefully that will help out places like CI that error on that repo missing. We can update it with the builds once cisco has them.

The F43 tarball was sent to the openh264-publishing@cisco.com mailing list with the title F43 RPMs for OpenH264 2.6.0 [1-5/5]. :thumbsup:

Please note that due to e-mail server file size limitations the tarball had to be split into 5 different parts, thus 5 e-mails in total.

I got confirmation from Cisco earlier today that they updated their CDN.

I verified it:
$ curl -I http://ciscobinary.openh264.org/openh264-2.6.0-2.fc43.x86_64.rpm
HTTP/1.1 200 OK

And went ahead and synced the repodata to sundries01.

I believe that should be it, let's see if something went wrong...

The rawhide packages seem to be signed with wrong key https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/PZNFCPDE5RMTQTWWNJLH7BPMZ6LYDFXP/

OpenPGP check for package "openh264-2.4.1-2.fc42.x86_64"
Those aren't the rawhide packages this ticket is about.

The RPMs that were just updated come from this build, openh264-2.6.0-2.fc43:
https://koji.fedoraproject.org/koji/buildinfo?buildID=2670904

for f40, f41, f42 and epel10 there are now builds for the newly release 2.5.1 version.

for f40, f41, f42 and epel10 there are now builds for the newly release 2.5.1 version.

Any particular reason for 2.5.1 instead of 2.6.0?

I have one more thing to ask of you.
Please never tag the builds into anything other than the dedicated *-openh264 tags (e.g. f43-openh264). If it's tagged into e.g. f43-testing-candidate it can eventually make its way to the main tag (e.g. f43) and the main buildroot, if not caught. This can cause very bad things to happen in terms of legal consequences.

Fortunately we managed to catch it in time and retagged the builds manually.

I am currently looking into how to make it so that this cannot happen in the first place, through tag policy right in Koji and then through a filter in Pungi. But this isn't done yet so we need to be very careful for now.

Thank you.

https://pagure.io/releng/issue/12635

Any particular reason for 2.5.1 instead of 2.6.0?

ABI bump... I think you do not want to attempt to coordinate this change between both Cisco and stable Fedoras! It's much less of an issue if rawhide is broken for a little while.

I have one more thing to ask of you.
Please never tag the builds into anything other than the dedicated *-openh264 tags (e.g. f43-openh264). If it's tagged into e.g. f43-testing-candidate it can eventually make its way to the main tag (e.g. f43) and the main buildroot, if not caught. This can cause very bad things to happen in terms of legal consequences.

@wtaymans please add a warning comment to the spec file immediately above the Name and Version, where the packager is unlikely to miss it, to demonstrate how to perform the builds with fedpkg build --target=f*-openh264. Otherwise, we'll forget this every time.

Any news on getting these builds to Cisco?

Given that https://fedoramagazine.org/announcing-fedora-linux-42-beta/ is out, it would be very nice to see codec there that doesn't freeze on seek.

Any news on getting these builds to Cisco?

Yes.

Given that https://fedoramagazine.org/announcing-fedora-linux-42-beta/ is out, it would be very nice to see codec there that doesn't freeze on seek.

F42 2.5.1 RPMs were sent to Cisco today. :thumbsup:

@patrikp for F42 I would expect to see 2.6.0

No, 2.6.0 is only for F43+.

Also, please note that https://koji.fedoraproject.org/koji/buildinfo?buildID=2667862 is still tagged f40-updates-candidate which means it is still possible to accidentally attach to an update.

I forgot. What is the right way to install https://koji.fedoraproject.org/koji/buildinfo?buildID=2677938 which is 2.6.0 for F41?

There is none. While there was some back and forth on this, ultimately 2.6.0 is not ABI compatible with 2.4.1/2.5.1, so it was limited to F43+.

There were 3 builds still tagged in a candidate tag, and I have untagged those.

Do note that due to the way things work, any packager can tag something into the candidate tags currently.
That should get fixed when we update policy to disallow it.

Alright, 2.5.1 F41/F40/EL10_1 tarballs were sent to Cisco earlier today (see the first comment in the ticket with the table).

However, after I sent the e-mails (they have to be split into 5 pieces each due to e-mail server size limitations) I got this:

For security reasons, Red Hat Mail does not allow you to use this type of file as it violates Google policy for executables and archives.

So it's unclear whether the archive actually got to them. I asked for confirmation and if they get back to me and say there is no attachment we'll have to find a different way to send it.

Happy days. :angry:

There's going to be no attachment.

How did we wind up in a world where we cannot email tarballs, I'm not sure. :(

@catanzaro Do you have any ideas about how we can actually get the RPMs to Cisco without legal problems? These licensing issues are the source of so much frustration. Ordinarily we'd just get the build and that's it. It's insane how many hoops we have to go through. But such is life I suppose...

CC: @kevin @yselkowitz

Probably need a secure drop URL of some kind to share with them.

Can they just send their signature that built RPM is good to go? In a blockchain.

Can they just send their signature that built RPM is good to go? In a blockchain.

I don't know what this means. What we have to do here is get the RPMs to Cisco somehow without allowing anybody except Cisco to download them.

I think easiest solution is to just try every possible archive type you can think of. They probably won't all be blocked? Alternatively, send them using personal mail instead of Red Hat email.

Could also try creating an IT ticket. If we can't fix Gmail to allow archives, then I'd say it's clearly no longer a suitable mail provider for engineers.

Probably need a secure drop URL of some kind to share with them.

Continuing @ngompa's line of thinking: Do we have a location where we could provide the builds for Cisco to pull? Or alternatively, could Cisco have a location for us to push to other than through email? Some things that come to mind (with the assumption these would have user tied auth and not allow CRUD except to those expected):

  • scp/sftp location
  • shared drive
  • https location
  • S3 or similar bucket

This would get us around the email issue and provide some amount of logging around what user did what, when. An email would still be required for coordination, but sending tarballs through email would no longer be required.

What we have to do here is get the RPMs to Cisco somehow without allowing anybody except Cisco to download them.

What is the point? The sources are open. If the builds are reproducible, they can build them themselfes, sign and publish signature.

Why the RPM should be sent in secret?

What we have to do here is get the RPMs to Cisco somehow without allowing anybody except Cisco to download them.

What is the point? The sources are open. If the builds are reproducible, they can build them themselfes, sign and publish signature.

Why the RPM should be sent in secret?

That's a very good question. I tried looking into it and it gave me a headache. In short, it's a licensing minefield.

OpenH264 is a free and open-source implementation of the H.264 video codec provided by Cisco. It's available both as source code and as a binary module. The source code is licensed under the Two-Clause BSD license, which is quite permissive and allows for use in both open-source and commercial projects.

Here's where things get a bit tricky. If we use the binary module distributed by Cisco, they cover the MPEG LA licensing fees for us. However, this binary must be downloaded at runtime from Cisco's servers. This means we can't just bundle the binary with Fedora; it needs to be fetched dynamically.

On the other hand, if you compile the source code yourself, you're responsible for paying any applicable licensing fees. This is because Cisco only covers fees for their own binary module, not for custom-built binaries.

So the workaround is that the (Cisco approved) binaries are built in Fedora's infrastructure but distributed by Cisco. We can never expose them to the public on our infrastructure because of some ancient patents.

The silver lining is that the last remaining patents should expire in August 2025 but this needs to be checked.

Update:
An e-mail was sent to a contact that we have over at Cisco. Let's see if we can put heads together and figure out a better way to get the RPMs over to them.

No reply to the last e-mail. But it was sent to only one person. I sent another e-mail and this time included more Cisco employees and the openh264-publishing mailing list in the hopes that someone might point us in the right direction.

If I don't get a reply in the next few days how will we move this forward? Would anybody happen to know a Cisco employee that might be able to help?

The openh264-publishing@ mailing list is how @lkocman and I interface with Cisco for openSUSE, and the responsiveness on that can be measured in days instead of weeks or months, so I think that should be good.

I've not seen any reply. Perhaps @patrikp has? Or we could perhaps resend?

No reply as of today, unfortunately. I'm not sure how to proceed here. We could re-send, yes, but I haven't got a reply to any of the other e-mails I sent either. I.e. when I sent the split tarballs some of them went through but there was no "part 1 is missing" kind of reply. We may need to try a different way to reach Cisco.

We are approaching 3 months since the publication of CVE-2025-27091. Due to the high risk level and inability to release updates, I fear we will need to just outright remove OpenH264 from Fedora.

We are painfully aware.

We finally have some contact with cisco folks, so I am hoping this will be unblocked soon.

Due to the high risk level and inability to release updates, I fear we will need to just outright remove OpenH264 from Fedora.

Also the lack of testing and performance issues urge users to install these non-free (or whatever) x264 codecs that (for some reason) are impossible to ship with Fedora.

Great news everyone (for a change)!

Cisco provided us with a very handy link for uploading.

The F40, F41, F42 and EPEL10 tarballs were successfully submitted.

Screenshot_from_2025-05-13_10-53-31.png

How much time it will take for the repos to update?

✗ sudo dnf5 update
Updating and loading repositories:
Repositories loaded.
Nothing to do.
✗ sudo dnf5 info openh264
Updating and loading repositories:
Repositories loaded.
Installed packages
Name            : openh264
Epoch           : 0
Version         : 2.4.1
Release         : 2.fc42
Architecture    : x86_64
Installed size  : 1.1 MiB
Source          : openh264-2.4.1-2.fc42.src.rpm
From repository : fedora-cisco-openh264
Summary         : H.264 codec library
URL             : https://www.openh264.org/
License         : BSD-2-Clause
Description     : OpenH264 is a codec library which supports H.264 encoding and decoding. It is
                : suitable for use in real time applications such as WebRTC.
Vendor          : Fedora Project

Cisco provided us with a very handy link for uploading.

Sounds great, as long as we're sure nobody else is able to use this link....

Hopefully they will get the updated packages on their CDN soon. As soon as they do we can update our repodata to point to them.

Sounds great, as long as we're sure nobody else is able to use this link....

What's the problem with that?

As soon as they do we can update our repodata to point to them.

That means Cisco tracks all Fedora users who run "dnf5 update"? That's mean.

What's the problem with that?

Well, we don't want random people pushing malware updates to Fedora users....

Well, we don't want random people pushing malware updates to Fedora users....

My bad. I missed "for uploading" part. For me giving some Fedora folk a part-time hourly paid position to handle files on the side of Cisco seems a more effective approach, cost wise too. I can just imagine how many high paid folks need to be distracted to handle this every time.

Can we have another status update here, please? I guess we're waiting on Cisco now?

Log in to comment on this ticket.

Metadata
Attachments 2