koji

#3995 Atomic tag/untag operations
Opened by alexi. Modified

Open
Close Issue

We have a hub plugin, similar to tag2distrepo, that is hooked into the postTag/postUntag callbacks and takes care of generating the repositories when a tag is changed. Requests for a single task are executed sequentially, as it doesn't make sense to try to generate the repository multiple times in parallel. This works fine, for the most part, but it can also result in temporarily inconsistent repositories.

Say you have two packages, one that depends on the other, and both of them must be upgraded together. Even if you tag them at the same time with a single CLI command, the hub will trigger two postTag callbacks. Once the execution of the first callback finishes, the repository will be in an inconsistent state until the second callback also finishes, as the repository will only include the first build and not the second.

It would be great if there was a way to have atomic tag/untag operations. Perhaps another pair of callbacks could be added for this, say preTagTransaction/postTagTransaction? This way, our plugin could be hooked into one of those instead, and handle one single request once all the builds are tagged/untagged.

The way to accomplish atomic tagging is perform the tags in a single api call. There will still be multiple callbacks, but everything will be in a single db transaction.

The easiest way to do this right now is to use the massTag api call. This requires that the user have the tag permission (which allows general editing and creation of tags).

There isn't a massUntag call, so for atomic untagging, the closest thing is to use multicall to perform multiple untagBuild calls in a single pass. There are a couple caveats though. While multicalls use a single db transaction, they don't stop on an error but instead roll back the one call and go on to the next one.

Note that he multicall approach won't work with the tagBuild because that call creates a task to do the work. You could use tagBuildBypass in a multicall, but that call requires the tag permission, so roughly equivalent to massTag which is the better option here.

Metadata Update from @mikem:
- Custom field Size adjusted to None

As I was writing the above, it occured to me that you are probably not so much concerned with database atomicity as with having some sort of atomic view from within the callbacks.

Unfortunately, I don't think that is reasonable to provide in general. Hub plugins are hub code, and internal to the system. As I wrote above, even if you make the tagging happen in a single db transaction, the callbacks will still be made.

I suspect that what you actually need is a change in your approach. I don't think you can handle all this regeneration logic correctly inside of individual callbacks. You need something more central. For non-dist-repos, this is handled in kojira, which is a separate service.

Edited by mikem

Metadata Update from @mikem:
- Issue tagged with: discussion

Actually, one thing you could do in your plugin is take the approach we use in the protonmsg plugin. We had a somewhat similar issue there where tag messages were sometimes being emitted before the tag was visible in the api. To work around that, the plugin saves up its messages each call and sends them during the postCommit callback.

So, if your entangled builds were being tagged atomically as described above, then your plugin could mimic the protonmsg approach and only actually trigger the regen during postCommit.

Yes, I think that approach could work for us, I'll look into it. I didn't realize there was a postCommit, it's not listed in https://docs.pagure.org/koji/writing_a_plugin/.

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/koji/koji/issues/3995

Please continue any further discussion there.
Metadata
None

discussion

None
None
Low
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.