fedora-ci / general

#56 RFE: rpmdiff as a service
Opened by churchyard. Modified

Open
Close Issue

I'd love to see rpmdiff output (possibly filtered by some config file) on a dist git Pull Request.

The service might work like this:

  • build the base and head of the PR (if possible, fallback to latest known build from Koji if only head builds)
  • run the RPMs trough rpmdiff
  • filter the output trough default (and customization) set of filters (regexes maybe?)
  • present the information in the PR (even via a comment)

(I realize that nobody probably has time for this, but if other people consider this useful, maybe I can get a student to work on this or something like that.)

@dcantrel Is it what the rpm-inspect does?

Thanks for the request. First, I want to make sure we are talking about the correct rpmdiff. There is an rpmdiff command that is part of the rpmlint project. But there is also an internal Red Hat tool called rpmdiff which is completely separate from the rpmlint project. Both rpmdiffs seek to discover differences between RPM packages. I work on the internal Red Hat rpmdiff.

I am working on is a new tool called rpminspect which is a refactoring of Red Hat's rpmdiff program. rpminspect is available here: https://github.com/dcantrell/rpminspect

I am working with @tflink on Fedora CI so we can run rpminspect on new builds from koji. The objective is to have Fedora CI running a variety of tests that provide feedback to the package maintainer and/or developer.

We're early in the development process, but if you have some students looking for some projects, I have a long list of tests to implement in rpminspect that would be good as individual projects. If you want more information there, send me an email.

I meant rpmdiff form rpmlint. However anything that can present the change is good.

For students, I'm looking for self contained Python projects that can be used as a semester work for our Python classes, most likely some integration project rather than internal logic of comparing RPMs.

Understood. rpminspect is implemented in C, but the higher level tooling around integrating it in to CI infrastructure and presenting results is likely in Python. @tflink has some prototype code at the moment in this area. Tim, do we have any part of the integration work where we could use some help for a student's semester?

Back on rpminspect, I have integration tests that need implementing in Python. There is a unit test suite and a separate integration test suite. To drive the integration test suite, I am using Python. Would working on that be useful for a semester project?

It might. The semester is not near yet, so I'm making a note not to forget about it.

sorry, the notification for this got lost in my inbox.

A bunch of the remaining work is waiting on decisions about what we're going to be doing going forward but if we can get through those, I think we could find things for a student to work on

We have rpminspect running against fedora currently, though the jenkins instance is having issues as noted here:

https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/thread/CRY6CU342EAUZOXXCXKELEQLQF35PEHK/

When running, it lives here currently:
http://fedora-build-checks.apps.ci.centos.org/

It's also something you can run locally at this point; the above link redirects, but this now lives at:

https://github.com/rpminspect/rpminspect

rpminspect tests are are non-blocking while we are in active development. I feel like we can safely close this ticket as development is still on-going, but if you disagree, please re-open! Thanks!
Edited by jimbair

Metadata Update from @jimbair:
- Issue status updated to: Closed (was: Open)

rpminspect does not (yet?) show how does the package differ from the previous version at all.

https://github.com/rpminspect/rpminspect/issues/26

Hence it does not solve this RFE.

See here what information is missing: https://github.com/rpminspect/rpminspect/issues/26#issuecomment-520217170

Metadata Update from @churchyard:
- Issue status updated to: Open (was: Closed)

It does show differences, but does not generate the output as mentioned in https://github.com/rpminspect/rpminspect/issues/26. That is an output mode I have on the to do list.

The output format in issue #26 for rpminspect also captures a lot of other pending tests that I need to port from rpmdiff. It's sort of an omnibus bug. I'm fine leaving that issue open. But we are now running rpminspect in fedora-ci and results are provided for the current inspections it performs.

It does show differences...

That's good news! Where can I see some example of differences it shows?

@churchyard if you'd like to see runs in fedora, those are over here:

http://fedora-build-checks.apps.ci.centos.org/

The readme over at https://github.com/rpminspect/rpminspect also shows you how to run rpminspect against one or two rpms; you can fetch releases for it over here:

https://copr.fedorainfracloud.org/coprs/dcantrell/rpminspect/

I have for example this Pull request:

https://src.fedoraproject.org/rpms/python-pip/pull-request/62

Or this update:

https://bodhi.fedoraproject.org/updates/FEDORA-2020-fd04d74051

Could you please navigate me to a place where I can see how the package differs from the previous version?

@tflink is this something you could quickly answer? If not, I can dig in and see what I can figure out. :)

This is a really needed functionality for essential packages that get merged into EL, thanks for working on this!

@tflink is this something you could quickly answer? If not, I can dig in and see what I can figure out. :)

Friendly bump.

As I understand we have rpminspect running in Zuul now. Where it needs to run in Fedora CI may have evolved since this ticket was opened. @mvadkert or @jimbair do you have any thoughts?

@dcantrell looking at the comments in this issue, seems @churchyard was mainly interested in the comparison run: https://github.com/rpminspect/rpminspect/issues/26#issuecomment-520217170

The rpminspect job runs on Zuul for a long time, but I still don't understand how it related to my RFE. I'd like to be able to see a summary of the differences between the PR and the actual latest Fedora build.

What files were changed/added/removed? What Provides/Requires/... were changed/added/removed? Etc. E.g I'd like to see something like:

$ rpmdiff ./auditwheel-4.0.0-1.fc36.noarch.rpm ./auditwheel-5.0.0-1.fc36.noarch.rpm 
removed     REQUIRES python3.10dist(setuptools)  
removed     PROVIDES auditwheel = 4.0.0-1.fc36
removed     PROVIDES python3.10dist(auditwheel) = 4
removed     PROVIDES python3dist(auditwheel) = 4
added       PROVIDES auditwheel = 5.0.0-1.fc36
added       PROVIDES python3.10dist(auditwheel) = 5
added       PROVIDES python3dist(auditwheel) = 5
..........T /usr/bin/auditwheel
..........T /usr/lib/python3.10/site-packages/auditwheel
removed     /usr/lib/python3.10/site-packages/auditwheel-4.0.0.dist-info
removed     /usr/lib/python3.10/site-packages/auditwheel-4.0.0.dist-info/INSTALLER
removed     /usr/lib/python3.10/site-packages/auditwheel-4.0.0.dist-info/LICENSE
removed     /usr/lib/python3.10/site-packages/auditwheel-4.0.0.dist-info/METADATA
removed     /usr/lib/python3.10/site-packages/auditwheel-4.0.0.dist-info/WHEEL
removed     /usr/lib/python3.10/site-packages/auditwheel-4.0.0.dist-info/entry_points.txt
removed     /usr/lib/python3.10/site-packages/auditwheel-4.0.0.dist-info/top_level.txt
added       /usr/lib/python3.10/site-packages/auditwheel-5.0.0.dist-info
added       /usr/lib/python3.10/site-packages/auditwheel-5.0.0.dist-info/INSTALLER
added       /usr/lib/python3.10/site-packages/auditwheel-5.0.0.dist-info/LICENSE
added       /usr/lib/python3.10/site-packages/auditwheel-5.0.0.dist-info/METADATA
added       /usr/lib/python3.10/site-packages/auditwheel-5.0.0.dist-info/WHEEL
added       /usr/lib/python3.10/site-packages/auditwheel-5.0.0.dist-info/entry_points.txt
added       /usr/lib/python3.10/site-packages/auditwheel-5.0.0.dist-info/pbr.json
added       /usr/lib/python3.10/site-packages/auditwheel-5.0.0.dist-info/top_level.txt
..........T /usr/lib/python3.10/site-packages/auditwheel/__init__.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__main__.py
..........T /usr/lib/python3.10/site-packages/auditwheel/__pycache__
..5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/__init__.cpython-310.opt-1.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/__init__.cpython-310.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/__main__.cpython-310.opt-1.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/__main__.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/condatools.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/condatools.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/elfutils.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/elfutils.cpython-310.pyc
added       /usr/lib/python3.10/site-packages/auditwheel/__pycache__/error.cpython-310.opt-1.pyc
added       /usr/lib/python3.10/site-packages/auditwheel/__pycache__/error.cpython-310.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/genericpkgctx.cpython-310.opt-1.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/genericpkgctx.cpython-310.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/hashfile.cpython-310.opt-1.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/hashfile.cpython-310.pyc
S.5.N.....T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/lddtree.cpython-310.opt-1.pyc
S.5.N.....T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/lddtree.cpython-310.pyc
added       /usr/lib/python3.10/site-packages/auditwheel/__pycache__/libc.cpython-310.opt-1.pyc
added       /usr/lib/python3.10/site-packages/auditwheel/__pycache__/libc.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/main.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/main.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/main_addtag.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/main_addtag.cpython-310.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/main_lddtree.cpython-310.opt-1.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/main_lddtree.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/main_repair.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/main_repair.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/main_show.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/main_show.cpython-310.pyc
added       /usr/lib/python3.10/site-packages/auditwheel/__pycache__/musllinux.cpython-310.opt-1.pyc
added       /usr/lib/python3.10/site-packages/auditwheel/__pycache__/musllinux.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/patcher.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/patcher.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/repair.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/repair.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/tmpdirs.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/tmpdirs.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/tools.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/tools.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/wheel_abi.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/wheel_abi.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/wheeltools.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/__pycache__/wheeltools.cpython-310.pyc
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/__init__.py
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/__pycache__
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/__pycache__/__init__.cpython-310.opt-1.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/__pycache__/__init__.cpython-310.pyc
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel
added       /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/LICENSE.txt
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/__init__.py
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/__pycache__
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/__pycache__/__init__.cpython-310.opt-1.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/__pycache__/__init__.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/__pycache__/pkginfo.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/__pycache__/pkginfo.cpython-310.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/__pycache__/util.cpython-310.opt-1.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/__pycache__/util.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/__pycache__/wheelfile.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/__pycache__/wheelfile.cpython-310.pyc
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/__init__.py
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/__pycache__
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/__pycache__/__init__.cpython-310.opt-1.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/__pycache__/__init__.cpython-310.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/__pycache__/convert.cpython-310.opt-1.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/__pycache__/convert.cpython-310.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/__pycache__/pack.cpython-310.opt-1.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/__pycache__/pack.cpython-310.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/__pycache__/unpack.cpython-310.opt-1.pyc
..5.......T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/__pycache__/unpack.cpython-310.pyc
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/convert.py
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/pack.py
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/cli/unpack.py
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/pkginfo.py
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/util.py
..........T /usr/lib/python3.10/site-packages/auditwheel/_vendor/wheel/wheelfile.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/condatools.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/elfutils.py
added       /usr/lib/python3.10/site-packages/auditwheel/error.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/genericpkgctx.py
..........T /usr/lib/python3.10/site-packages/auditwheel/hashfile.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/lddtree.py
added       /usr/lib/python3.10/site-packages/auditwheel/libc.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/main.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/main_addtag.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/main_lddtree.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/main_repair.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/main_show.py
added       /usr/lib/python3.10/site-packages/auditwheel/musllinux.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/patcher.py
..........T /usr/lib/python3.10/site-packages/auditwheel/policy
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/policy/__init__.py
..........T /usr/lib/python3.10/site-packages/auditwheel/policy/__pycache__
S.5.N.....T /usr/lib/python3.10/site-packages/auditwheel/policy/__pycache__/__init__.cpython-310.opt-1.pyc
S.5.N.....T /usr/lib/python3.10/site-packages/auditwheel/policy/__pycache__/__init__.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/policy/__pycache__/external_references.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/policy/__pycache__/external_references.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/policy/__pycache__/versioned_symbols.cpython-310.opt-1.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/policy/__pycache__/versioned_symbols.cpython-310.pyc
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/policy/external_references.py
added       /usr/lib/python3.10/site-packages/auditwheel/policy/manylinux-policy.json
added       /usr/lib/python3.10/site-packages/auditwheel/policy/musllinux-policy.json
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/policy/policy-schema.json
removed     /usr/lib/python3.10/site-packages/auditwheel/policy/policy.json
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/policy/versioned_symbols.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/repair.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/tmpdirs.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/tools.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/wheel_abi.py
S.5.......T /usr/lib/python3.10/site-packages/auditwheel/wheeltools.py
..........T /usr/share/doc/auditwheel
S.5.......T /usr/share/doc/auditwheel/README.rst
$ rpmdiff ./auditwheel-4.0.0-1.fc36.src.rpm ./auditwheel-5.0.0-1.fc36.src.rpm 
removed     REQUIRES python3dist(setuptools)  
removed     REQUIRES python3dist(tox) >= 1.6
removed     REQUIRES python3dist(tox-current-env) >= 0.0.6
added       REQUIRES python3dist(setuptools) >= 42
removed     PROVIDES auditwheel = 4.0.0-1.fc36
added       PROVIDES auditwheel = 5.0.0-1.fc36
removed     319.patch
removed     auditwheel-4.0.0.tar.gz
added       auditwheel-5.0.0.tar.gz
S.5.......T auditwheel.spec

Possibly with a way to hide some of this, when it gets noisy, as we do in rpmlintrc file for rpmlint.

Maybe Zuul can gain an rpmdiff job, similarily how it already has a rpmlint job?

Edited by churchyard

With the are of dynamic BuildRequires and generated Requires, it would be really helpful to see that a simple "update to a newer version" PR added a new (possibly undesired) build dependency. I can check that manually, but with the number of changes, having that automated would be extremely helpful.

Ahhh, I see. Yes, this is definitely something I have had on my to do list for a long time. I just need to add a new output option to rpminspect. I restructured the results data so that I can more easily output a format like you describe, I just need to add the function(s).

Can I call the new output format "miro"? :)

I suggest calling it "rpmdiff" (or if that is too ambiguous, "rpmlint-rpmdiff").

I don't want to use the name of an existing tool for the output mode and I don't want to say "rpmdiff" anywhere in rpminspect.

Let's continue the rpminspect-specifics in this issue:
https://github.com/rpminspect/rpminspect/issues/26

https://github.com/rpminspect/rpminspect/issues/26 is closed. can we please see this summery view on the CI?

Hmm, @msrb ?

So since this ticket was opened, rpminspect has also gained a results viewer that can be integrated in to web frontends. I was hoping this could be something Zuul CI could use to display job results. There may need to be some tweaks done.

Here's the PR where that contribution originated:
https://github.com/rpminspect/rpminspect/pull/905

The images in https://github.com/rpminspect/rpminspect/pull/905 look nice but neither of them appears to be showing the differences between the new and old build in the way rpmlint's rpmdiff does, which is all I really ever wanted in this ticket :/

The images in https://github.com/rpminspect/rpminspect/pull/905 look nice but neither of them appears to be showing the differences between the new and old build in the way rpmlint's rpmdiff does, which is all I really ever wanted in this ticket :/

I know, and I haven't forgotten. There's been very little interest in having anything like rpmlint output, but I still want to get that working. I've never really used rpmlint either, so this is something I need to spend some time doing. I just wanted to point out the new viewer, which I think will be a nice addition to the Zuul UI.
Metadata
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.