#4342 download-build: allow fallback to unsigned with --key
Opened 2 months ago by adamwill. Modified 4 days ago

file modified
+24 -2
@@ -19,6 +19,7 @@ 

  from dateutil.tz import tzutc

  from optparse import SUPPRESS_HELP, OptionParser

  

+ from requests.exceptions import HTTPError

  import six

  import six.moves.xmlrpc_client

  from six.moves import filter, map, range, zip
@@ -6830,6 +6831,8 @@ 

      parser.add_option("--task-id", action="store_true", help="Interperet id as a task id")

      parser.add_option("--rpm", action="store_true", help="Download the given rpm")

      parser.add_option("--key", help="Download rpms signed with the given key")

+     parser.add_option("--fallback-unsigned", action="store_true",

+                       help="When used with --key: download unsigned if signed packages not found")

      parser.add_option("--topurl", metavar="URL", default=options.topurl,

                        help="URL under which Koji files are accessible")

      parser.add_option("--noprogress", action="store_true", help="Do not display progress meter")
@@ -6912,6 +6915,7 @@ 

                  continue

              rpms.append(rpm)

  

+     unsigned = []

      if suboptions.key:

          with session.multicall() as m:

              results = [m.queryRPMSigs(rpm_id=r['id'], sigkey=suboptions.key) for r in rpms]
@@ -6921,14 +6925,32 @@ 

                  nvra = "%(nvr)s-%(arch)s.rpm" % rpm

                  warn("No such sigkey %s for rpm %s" % (suboptions.key, nvra))

                  rpms.remove(rpm)

+                 if suboptions.fallback_unsigned:

+                     unsigned.append(rpm)

  

-     size = len(rpms) + len(archives)

+     size = len(rpms) + len(unsigned) + len(archives)

      number = 0

  

      # run the download

      for rpm in rpms:

          number += 1

-         download_rpm(info, rpm, suboptions.topurl, sigkey=suboptions.key, quiet=suboptions.quiet,

+         try:

+             download_rpm(info, rpm, suboptions.topurl, sigkey=suboptions.key, quiet=suboptions.quiet,

+                          noprogress=suboptions.noprogress, num=number, size=size)

+         except HTTPError as err:

+             # this is necessary even with the 'unsigned' handling above

+             # because sometimes queryRPMSigs will still tell us a

+             # package was signed with a given key, but the signed copy

+             # has been garbage-collected

+             if suboptions.key and suboptions.fallback_unsigned and err.response.status_code == 404:

+                 warn("Signed copy not present, will download unsigned copy")

+                 download_rpm(info, rpm, suboptions.topurl, sigkey=None, quiet=suboptions.quiet,

+                              noprogress=suboptions.noprogress, num=number, size=size)

+             else:

+                 raise

+     for rpm in unsigned:

+         number += 1

+         download_rpm(info, rpm, suboptions.topurl, sigkey=None, quiet=suboptions.quiet,

                       noprogress=suboptions.noprogress, num=number, size=size)

      for archive in archives:

          number += 1

If you pass --key to download-build and signed packages aren't
available, Koji will skip the unsigned package, or error out.
This adds a modified behavior controlled by the new
--fallback-unsigned arg. If this is passed with --key, unsigned
copies will be downloaded for packages for which no signed copy
can be found.

This is primarily intended to work with a proposed Bodhi feature:
https://github.com/fedora-infra/bodhi/pull/5859 . That would
make Bodhi's bodhi updates download command automatically try
to download signed copies, but I think it would be best if it
falls back to getting unsigned copies if that doesn't work. Just
failing out entirely seems wrong for that case. Implementing the
fallback in Bodhi itself is more awkward and messy than adding it
in Koji, and it may be useful for others in Koji I guess.

Note there are two distinct 'no signed copies' cases. In the
simple one, queryRPMSigs tells us Koji has no record of the
package ever being signed with the key in question. In this case
we don't bother trying to download a signed copy. In the other
case, queryRPMSigs tells us the package has been signed with
the key, but it turns out that signed copy has been garbage-
collected and we can no longer download it. In this case we have
to catch the failure on the download attempt and retry the
download with sigkey set to None.

Signed-off-by: Adam Williamson awilliam@redhat.com

Metadata Update from @tkopecek:
- Pull-request tagged with: testing-ready

a month ago

This options seems fine.

It seems like this command could benefit from a --write-signed-rpms option, but that can be separate work (and since it requires a special perm, probably not that useful for the use-case above)

Metadata Update from @tkopecek:
- Pull-request untagged with: testing-ready
- Pull-request tagged with: testing-basic

a month ago

Are you waiting on anything from me here?

We're waiting on qe testing

Metadata Update from @mfilip:
- Pull-request tagged with: testing-done

4 days ago