Describe the issue Now PDC is gone, we don't have a store of metadata from old composes. We don't strictly need one, but it is useful info sometimes, and it's small (the RPMs file is ~150M uncompressed, the others are less than 100K). It should be fairly easy to copy the metadata files to a different directory after the compose completes, and not garbage-collect that directory, so it functions as such a store.
When do you need this? (YYYY/MM/DD) No specific date.
When is this no longer needed or useful? (YYYY/MM/DD) When there's some other way of accessing old compose data, I guess.
If we cannot complete your request, what is the impact? It will not be possible to get useful data on composes that have been garbage collected.
As a bonus, it would be nice to 'import' the image metadata for all past composes that's still in PDC's database, though PDC did modify it a bit on import so we would need to write a script to unconvert it back to the images.json format, I think.
So my thought on implementation here is to just add some stuff to nightly.sh.
That would copy the compose info to something like /mnt/koji/compose/rawhide/md/YYYY/MM/DD/composeid/ (md for metadata)
We could also compress it? but I guess it doesn't matter much?
I am unsure if we could extract the old pdc data, but if we have cycles to try we could. At this point it would be directly poking the database I guess.
Metadata Update from @phsmoura: - Issue tagged with: low-trouble, medium-gain, ops
From last nights run:
cp: cannot stat '/mnt/koji/compose/rawhide/Fedora-Rawhide-20240813.n.0/compose/metadata/*.json': No such file or directory ./nightly.sh: line 247: /mnt/koji/compose/metadata-archive/series/Fedora-rawhide: No such file or directory
Log in to comment on this ticket.