Learn more about these different git repos.
Other Git URLs
As part of PDC retirement rewrite logic to stop querying PDC for required information. The source of truth if the package is retired should be the presence of dead.package on its release branch in dist-git. The source of truth about active release branches should be bodhi API.
./scripts/block_retired.py ./scripts/pdc/get-retired-packages.py ./scripts/fedretire
Metadata Update from @lenkaseg: - Issue assigned to lenkaseg
Related #11757
Proposed solution:
There are three scripts in releng that check for retired packages, not package by package, but all retired at once. PDC has the information about the retired packages (active=False) because the information about a retired package is sent to PDC at the event of retiring. Without PDC there would be no place to have names of all retired packages stored.
active=False
We can extract the information about retired packages from /srv/git/rpms/*.git on pkgs01 machine - those that contain dead.package(checked or each branch).
/srv/git/rpms/*.git
pkgs01
dead.package
There is a simple bash script that would retrieve the retired packages per branch (#PR11757) nightly and output a json in this format: {"branch1": [package01, package02]¸branch2: [package01, package02]}. The json would then be exposed on some API endpoint somewhere (distgit?). Releng scripts would then check the retired packages there.
First thanks for working on this, sorry for the delay in feedback. ;(
This seems like it would work, but oof... thats a LOT of io. We have 38262 packages x (right now) 4 active fedora branches x ( right now) 4 active epel branches, which means we need to check something like 306096 branches... and per day? that would put a ton of load on pkgs01. ;(
So, perhaps we could ponder on what those scripts do and if we can figure a better way to do it.
block_retired.py is run even day with the rawhide compose to block packages in koji that were retired in the last day. If we move this retirement to a toddler, we don't need to run this script anymore?
I am not sure what the other two are for. @humaton any ideas?
would having the equivalent data stored in bodhi instead of pdc work? I think this was one of the items that was flagged as something bodhi could do as it sort of covered things it already did?
Perhaps.
Also the dead.package commits are all in datagrepper / on the fedora-messaging bus... but I don't think it's super easy to get that information, you have to get all commit messages and unpack them to see if dead.package was created as part of the commit. ;(
So I played with the script Lenka created. It took:
real 18m47.324s user 3m25.859s sys 10m47.095s
I didn't notice any particular spike in load, but monitoring might have.
We will extend this toddler in the process of retiring PDC. https://pagure.io/fedora-infra/toddlers/blob/main/f/toddlers/plugins/pdc_retired_packages.py
after the toddlers update, there will be no need to keep running the block_retired.py.
Related PR on ansible which shold schedule the script nightly on pkgs01. https://pagure.io/fedora-infra/ansible/pull-request/1650#
Related PR #11796, removes pdc from get-retired-packages.py https://pagure.io/releng/pull-request/11796
get-retired-packages.py
I didn't find fedretire neither in ansible nor toddlers. Is there any other place where it could be used?
fedretire
Related PR#12024 Fedretire script removed, it's not used anymore
This issue can be considered fixed when the block-retired.py script is solved.
block-retired.py
@amedvede Would you have an update on block-retired.py please?
@amedvede's PR of a toddler which blocks retired packages in koji is merged, maybe we could close this issue? @humaton
https://pagure.io/fedora-infra/toddlers/pull-request/201
Most of the work here is completed, one quick thing is to clean it up from the nightly,sh (60 and 61 line)
https://pagure.io/pungi-fedora/blob/main/f/nightly.sh#_60
Metadata Update from @jnsamyak: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.