releng

Clone

#11129 distgit-branch-unused: ignore infra builds which don't specify a source
Opened 2 years ago by zbyszek. Modified 3 months ago
zbyszek/releng distgit-branch-unused-infra  into  main

Merge
file modified
+19 -4 
@@ -99,6 +99,8 @@ 
 

      return False
 

  

  def find_hash(build):
 

+     if '#' not in build['source']:
 

+         return None
 

      return build['source'].rsplit('#', 1)[1]
 

  

  def list_builds(package, opts):
 
@@ -117,18 +119,31 @@ 
 

          for build in builds:
 

              if build['source'] is None:
 

                  build['source'] = msession.getTaskInfo(build['task_id'], request=True)
 

+ 

+     by_hash = {}
 

      for build in builds:
 

          if isinstance(build['source'], koji.VirtualCall):
 

              r = build['source'].result
 

              if r is None:
 

                  # This seems to happen for very old builds, e.g. buildbot-0.7.5-1.fc7.
 

-                 build['source'] = None
 

                  nvr, time = build['nvr'], build['creation_time']
 

                  print(f'Warning: build {nvr} from {time} has no source, ignoring.')
 

-             else:
 

-                 build['source'] = r['request'][0]
 

+                 continue
 

+ 

+             build['source'] = r['request'][0]
 

+ 

+         if build['source']:
 

+             hash = find_hash(build)
 

+             if hash is None:
 

+                 nvr, time, source = build['nvr'], build['creation_time'], build['source']
 

+                 if nvr.endswith('infra'):
 

+                     print(f'Warning: infra build {nvr} from {time} has source {source!r} with no hash, ignoring.')
 

+                     continue
 

+ 

+                 raise ValueError(f'Warning: build {nvr} from {time} has source {source!r} with no hash')
 

+ 

+             by_hash[hash] = build
 

  

-     by_hash = {find_hash(b):b for b in builds if b['source']}
 

      return by_hash
 

  

  def mbs_builds_exist(builds):

I found yet another case where source build information is lacking. For the
python-rpmautospec python pakcage there is a number of builds with 'source'
specified as 'python-rpmautospec-0.0.13-1.el7.infra.src.rpm', i.e. the package
was not built from dist-git directly. If the package version ends with 'infra',
assume that the build was done by infra and ignore those builds during branch
checks. Otherwise, throw an error.

With this, it is possible to again possible to check a branch for removal:

$ scripts/distgit-branch-unused.py --repository /tmp/spora/home/zbyszek/fedora/python-rpmautospec origin/convert-to-rpmautospec
Checking package python-rpmautospec in /tmp/spora/home/zbyszek/fedora/python-rpmautospec
Branch convert-to-rpmautospec not found in bodhi, checking if branch matches pattern...
...no match, seems OK to remove
Querying koji for builds...
Warning: infra build python-rpmautospec-0.0.13-1.el7.infra from 2020-04-07 10:07:48.794816+00:00 has source 'python-rpmautospec-0.0.13-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.13-1.fc31.infra from 2020-04-07 10:05:18.560493+00:00 has source 'python-rpmautospec-0.0.13-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.14-1.fc31.infra from 2020-04-07 10:22:19.593743+00:00 has source 'python-rpmautospec-0.0.14-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.14-1.el7.infra from 2020-04-07 10:24:14.030490+00:00 has source 'python-rpmautospec-0.0.14-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.12-1.el7.infra from 2020-04-07 09:26:34.964569+00:00 has source 'python-rpmautospec-0.0.12-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.12-1.fc31.infra from 2020-04-07 09:26:41.411434+00:00 has source 'python-rpmautospec-0.0.12-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.15-1.el7.infra from 2020-04-07 12:49:50.148921+00:00 has source 'python-rpmautospec-0.0.15-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.15-1.fc31.infra from 2020-04-07 12:50:49.266406+00:00 has source 'python-rpmautospec-0.0.15-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.1-1.fc31.infra from 2020-03-31 12:51:39.168110+00:00 has source 'python-rpmautospec-0.0.1-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.1-1.el7.infra from 2020-03-31 13:12:21.154392+00:00 has source 'python-rpmautospec-0.0.1-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.16-1.fc31.infra from 2020-04-07 15:22:13.490178+00:00 has source 'python-rpmautospec-0.0.16-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.16-1.el7.infra from 2020-04-07 15:21:43.214562+00:00 has source 'python-rpmautospec-0.0.16-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.17-1.el7.infra from 2020-04-07 16:15:21.073787+00:00 has source 'python-rpmautospec-0.0.17-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.17-1.fc31.infra from 2020-04-07 16:16:00.145873+00:00 has source 'python-rpmautospec-0.0.17-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.2-1.el7.infra from 2020-03-31 13:57:35.488347+00:00 has source 'python-rpmautospec-0.0.2-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.2-1.fc31.infra from 2020-03-31 13:57:46.110492+00:00 has source 'python-rpmautospec-0.0.2-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.4-1.fc31.infra from 2020-03-31 16:44:06.262229+00:00 has source 'python-rpmautospec-0.0.4-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.4-1.el7.infra from 2020-03-31 16:46:53.463456+00:00 has source 'python-rpmautospec-0.0.4-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.5-1.el7.infra from 2020-03-31 18:18:56.700853+00:00 has source 'python-rpmautospec-0.0.5-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.18-1.fc31.infra from 2020-04-08 08:07:05.439316+00:00 has source 'python-rpmautospec-0.0.18-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.18-1.el7.infra from 2020-04-08 08:11:46.691931+00:00 has source 'python-rpmautospec-0.0.18-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.21-1.el7.infra from 2020-04-08 09:16:53.284453+00:00 has source 'python-rpmautospec-0.0.21-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.21-1.fc31.infra from 2020-04-08 09:17:31.983737+00:00 has source 'python-rpmautospec-0.0.21-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.11-1.fc31.infra from 2020-04-06 14:42:50.860762+00:00 has source 'python-rpmautospec-0.0.11-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.11-1.el7.infra from 2020-04-06 14:45:51.479521+00:00 has source 'python-rpmautospec-0.0.11-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.19-1.el7.infra from 2020-04-08 08:41:03.543316+00:00 has source 'python-rpmautospec-0.0.19-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.1.3-1.el7.infra from 2020-04-09 13:24:30.402596+00:00 has source 'python-rpmautospec-0.1.3-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.1.3-1.fc31.infra from 2020-04-09 13:25:17.747225+00:00 has source 'python-rpmautospec-0.1.3-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.1.0-1.el7.infra from 2020-04-09 12:06:00.423067+00:00 has source 'python-rpmautospec-0.1.0-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.1.0-1.fc31.infra from 2020-04-09 12:05:34.675958+00:00 has source 'python-rpmautospec-0.1.0-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.1.2-1.el7.infra from 2020-04-09 12:09:00.819906+00:00 has source 'python-rpmautospec-0.1.2-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.1.2-1.fc31.infra from 2020-04-09 12:09:47.031267+00:00 has source 'python-rpmautospec-0.1.2-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.20-1.el7.infra from 2020-04-08 08:51:14.683771+00:00 has source 'python-rpmautospec-0.0.20-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.20-1.fc31.infra from 2020-04-08 08:51:33.372885+00:00 has source 'python-rpmautospec-0.0.20-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.22-1.el7.infra from 2020-04-08 16:19:02.631521+00:00 has source 'python-rpmautospec-0.0.22-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.22-1.fc31.infra from 2020-04-08 16:19:45.224605+00:00 has source 'python-rpmautospec-0.0.22-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.23-1.el7.infra from 2020-04-08 19:27:42.833532+00:00 has source 'python-rpmautospec-0.0.23-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.23-1.fc31.infra from 2020-04-08 19:28:23.018048+00:00 has source 'python-rpmautospec-0.0.23-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.7-1.fc31.infra from 2020-04-03 14:09:39.340186+00:00 has source 'python-rpmautospec-0.0.7-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.7-1.el7.infra from 2020-04-03 14:10:10.015570+00:00 has source 'python-rpmautospec-0.0.7-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.8-1.el7.infra from 2020-04-03 15:10:36.905991+00:00 has source 'python-rpmautospec-0.0.8-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.8-1.fc31.infra from 2020-04-03 15:11:41.637107+00:00 has source 'python-rpmautospec-0.0.8-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.9-1.el7.infra from 2020-04-03 16:04:36.877078+00:00 has source 'python-rpmautospec-0.0.9-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.9-1.fc31.infra from 2020-04-03 16:04:27.050746+00:00 has source 'python-rpmautospec-0.0.9-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.3-1.fc31.infra from 2020-03-31 15:37:31.381552+00:00 has source 'python-rpmautospec-0.0.3-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.6-1.fc31.infra from 2020-04-02 15:18:00.293237+00:00 has source 'python-rpmautospec-0.0.6-1.fc31.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.6-1.el7.infra from 2020-04-02 15:18:17.946926+00:00 has source 'python-rpmautospec-0.0.6-1.el7.infra.src.rpm' with no hash, ignoring.
Warning: infra build python-rpmautospec-0.0.5-1.fc31.infra from 2020-03-31 18:15:18.870708+00:00 has source 'python-rpmautospec-0.0.5-1.fc31.infra.src.rpm' with no hash, ignoring.
Branch has commits not found anywhere else, checking builds...
0: 415e1d2 Convert to rpmautospec
1: 80b09cd Update to 0.3.0
Commit 80b09cdf6ed2ee3cb771a91dbfb6a1c54a5eef5d referenced from refs/remotes/my/convert-to-rpmautospec, refs/remotes/origin/HEAD, refs/remotes/origin/epel9, refs/remotes/origin/f35, refs/remotes/origin/f36, refs/remotes/origin/f37, refs/remotes/origin/main, refs/remotes/origin/rawhide. Stopping iteration.
No builds found, seems OK to delete.

Signed-off-by: Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl

I suppose this is reasonable, but could you instead find anything that was built from a src.rpm / no hash and toss that?

I mean, it should really only be the infra tags, but in theory there could be some other non production tag where we allow build from src.rpm instead of scm.

So, I'd be ok merging this, but it would be nice if it was more generic.

Any thoughts on the above questions?

I agree with the question, i.e. I think it'd be nice to rework the PR. I was just short on time, sorry. I plan to return to this at some point, but I have no idea when.

ok, no problem at all. :)

Any news here?
:)

Status is unchanged. If you tell me that this would be useful to have, i.e. that there are requests where this actually matters, I'd try make some space for this.

I don't think it's any kind of urgent, but it would be nice to have I think... we can leave it for some day when more time exists.

Metadata
None
No Tags
Changes Summary 1
+19 -4
file changed
scripts/distgit-branch-unused.py
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.