#8390 Create custom git refs to archive deprecated branches in dist-git
Closed: Fixed 3 years ago by pingou. Opened 4 years ago by bookwar.

The issue of dist-git having "too many branches" regularly appears on mailing lists and in offline conversations.

I'd like to propose a solution to this problem which uses custom git references.

In Git branches and tags are just two special cases of a git ref, stored under .git/refs/heads/
and .git/refs/tags/ respectively.

These refs are treated in a special way by the high level tools, but internally these are the same thing. Because tags and branches have special meaning they are made visible in CLI and GUI.

Let's create additional ref namespace called archive (.git/refs/archive) and move deprecated branch references to it.

  • Since ref will be preserved, git garbage collection will not remove the git commits which they point to.
  • Since it is a custom namespace, these branches will not be exposed to the user by default.
  • Since we change namespace, not the branch name, it will be easy to find the old branch after archiving.

User will still be able to fetch those references, by requesting them through a fully quailified path: Instead of running git checkout old-branch one would need to use git checkout refs/archive/old-branch.


on the technical side this is not a big deal, we could add a pagure-admin archive-branch project branchor an api endpoint or whatever depending on how are we going to apply it, but shouldn't we change the policy first?

I agree, should be doable, both through Pagure API and directly via git.

1) I want to get the confirmation that there are no technical obstacles to this. Maybe some infrastructure dependencies which I am not aware of.

2) On the policy side, for me it doesn't look like we are changing it. We don't remove old branches, we just reorganize them. So it is not a policy change, but we would need to update some docs to explain how it can be used.

What happens if one then creates a new branch with the same name, and then wants to archive the new branch, too?

What happens if one then creates a new branch with the same name, and then wants to archive the new branch, too?

git pre-receive hook denying such branches should help

I guess you'll want a command to unarchive an archived branch then, so that the name is not lost forever if you accidentally archive a branch.

I guess you'll want a command to unarchive an archived branch then, so that the name is not lost forever if you accidentally archive a branch.

Possibly. But this should be done just on request. The question is if you want to recycle branch names for different purposes then originally.

What happens if one then creates a new branch with the same name, and then wants to archive the new branch, too?

Thanks, I haven't thought about it.

One option is a githook as Vit said. The other - to change the proposal slightly and add a timestamp or counter to the archived branch name

git checkout refs/archive/old-branch/20190815

This will mess up with the discoverability of an archived branch a bit, but it provides a possibility to reuse branch names for some other purpose, and archive them several times.

Although, since our branches are not totally random, and usually are restricted by other policies so they don't overlap, the hook seems to be a much cleaner option.

What branches were you picturing as being archived? Only requested by maintainers? All EOL branches as they EOL?

Do you know the git operations to make this archive namespace and move branch refs to it off hand? (ie, we will need some kind of script to do this at EOL or on request or whatever).

Do you know the git operations to make this archive namespace and move branch refs to it off hand? (ie, we will need some kind of script to do this at EOL or on request or whatever).

I was thinking on creating the refs/archive pointing to the same obj that refs/heads points and removing the refs/heads entry after with an pagure-admin subcommand or a pagure-dist-git script

What about visibility? Should we show them someway on src.fp.o ?

This was the trigger for this ticket:

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/T66D6KPT5KOZKDRBMRVKCMECUDW2PBAA/

So obviously, I would like to see removed all the modularity branches, which are not used (but TBH, I was more thinking about something similar to dead.package we have for retired packages). I would not mind if the regular Fnn branches after their EOL were removed as well.

And of course I would like to be able to get rid of any other branch I am able to create, but not allowed to delete.

May be there could also be more namespaces, depending what was the reason for "removal"

Metadata Update from @mizdebsk:
- Issue tagged with: src.fp.o

4 years ago

So, I am still not sure of the scope here. Is it:

a) when branches are retired or eoled, we script something to move those branches to archive/ when they are eol or archived. Users can request other branches be archived via a releng ticket.

or

b) Users are allowed to move any branches to archive they like.

or something else?

If it's a, we just need to scope the work on making the scripts and adjusting eol and retire processes.

If it's b, we may need to see if our hook setup would allow that (ie, only moving a ref from heads to archive, not allowing delete?

Metadata Update from @kevin:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

4 years ago

So, I am still not sure of the scope here.

a) is the scope. I don't think users can move branches or should even be able to move them.

However it needs to be supported by some git hooks to prevent pushes into archived branches, pushing branches with colliding names, etc.

So, I am still not sure of the scope here.

a) is the scope. I don't think users can move branches or should even be able to move them.

:thumbsup:

However it needs to be supported by some git hooks to prevent pushes into archived branches, pushing branches with colliding names, etc.

push to retired branches is denied already[0] but... is it possible to push refs to refs/archive with git? No idea really. If its possible, we'll need small tweaks to deny pushing to refs/archive/* and we should decide wether we make the basename(refname) unique denying reusing of names on refs/archive or if we allow to reuse them and archive with refs/archive/data-name or something similar to avoid collisions.

0: https://pagure.io/pagure-dist-git/blob/master/f/dist_git_auth.py#_82-117

Metadata Update from @pingou:
- Issue priority set to: Next Meeting (was: Waiting on Assignee)

3 years ago

So, looking back at this again...

Would improvements in the pagure/dist-git UI solve this?

Right now it lists branches that are active as bold and inactive ones greyed out, but it's pretty hard to notice I think.
If we split that page to list 'active' and then 'inactive' branches in two seperate sections would that solve the issue?

I think that would be much easier than modifying all the git repos to archive things.

Thoughts?

@kevin it would be better than nothing, but it won't help working with actual repository (not mentioning the discussed move to GitLab).

Well moving heads to archives would actually make the migration to gitlab harder if anything

But what about the combo box in commits history for example? And the $ git branch output?

git branch by default shows only the local branches, so you could simply git branch -D f30 to no longer have it show in git branch.

Commit de6f7f49 relates to this ticket

Commit https://pagure.io/fedora-infrastructure/c/de6f7f49 relates to this ticket

The commit is probably from different repo, right?

The commit is probably from different repo, right?

Yes, it's in the pagure repo, I've just fixed the link

Metadata Update from @smooge:
- Issue tagged with: dev, medium-gain, medium-trouble

3 years ago

@pingou do you know when this can land on src?

@smooge the new branches page will be in 5.12 which I hope to get out soon (but is kinda block by the pagure-messages integration PR)

The new branch page has now been deployed: https://src.fedoraproject.org/rpms/guake/branches?branchname=master

I'm going to close this ticket as fixed and we can always re-open if you disagree or believe it needs more discussions.

Thanks for your inputs everyone!

Metadata Update from @pingou:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.

Metadata
Boards 1
dev Status: Done
Attachments 1
Related Pull Requests
  • #5008 Merged 3 years ago