From 5449f0bb38f88b6afa46472d6a6614cf384e41a7 Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Mar 04 2019 21:56:58 +0000 Subject: [PATCH 1/9] bump release to 1.17.0 --- diff --git a/koji.spec b/koji.spec index 022e2aa..91c11e6 100644 --- a/koji.spec +++ b/koji.spec @@ -81,7 +81,7 @@ %define release %{baserelease} %endif Name: koji -Version: 1.16.0 +Version: 1.17.0 Release: %{release}%{?dist} License: LGPLv2 and GPLv2+ # the included arch lib from yum's rpmUtils is GPLv2+ @@ -713,6 +713,118 @@ fi %endif %changelog +* Thu Feb 28 2018 Mike McLean - 1.17.0-1 +- PR#1227: Added volume id as argument to livemedia and livecd tasks +- PR#1070: consolidate access to rpm headers +- PR#1274: cve-2018-1002161 +- PR#1271: decode Popen.communicate result under py3 +- PR#1269: require librepo on python3 +- PR#1222: Include CLI plugins in setup.py +- PR#1265: py3 tests + related fixes +- PR#1220: Fix non-ascii strings in xmlrpc +- PR#1229: document reason strings in policies +- PR#1263: python 3 can't index dict.keys() +- PR#1235: fix weak deps handling in rpminfo web page +- PR#1251: fix race-condition with librepo temp directories +- PR#1245: organize python 2/3 cases in spec file +- PR#1231: remove unused directory +- PR#1248: use six move for email.MIMEText +- PR#1150: using ConfigParser.read_file for PY3 +- PR#1249: more detailed help for block-group-pkg +- PR#1117: python3 kojid +- PR#891: Web UI python3 changes +- PR#921: Py3 hub +- PR#1182: hub: document get_channel arguments +- PR#1014: cli: preserve build order in clone-tag +- PR#1218: docs: drop HTML tags from howto doc +- PR#1211: Fix wrong error message +- PR#1184: rest of python3 support for koji lib +- PR#1062: fix pyOpenSSL dependency for py26 in setup.py +- PR#1019: Use python2/3 instead of python in Makefile/spec +- PR#1190: hub: document all edit_tag arguments +- PR#1201: re-add urlparse import in kojikamid +- PR#1203: Fix `is_conn_error()` for Python 3.3+ change to `socket.error` +- PR#967: use correct fileinfo checksum field +- PR#1187: Add ctx option to ClientSession.krb_login() +- PR#1175: kojira: avoid race condition that causes "unknown task" errors +- PR#964: few sort speedups +- PR#852: drop encode_int helper +- PR#1043: remove old messagebus plugin +- PR#1176: kojid: implement task_avail_delay check +- PR#1180: Update source when recycling build +- PR#1178: cli: document parse_arches method parameters +- PR#920: use relative symlinks for hub imports +- PR#981: cli: add a param in watch_tasks to override KeyboardInterrupt output +- PR#1042: don't fail on missing field in base policy tests +- PR#1172: make timeout of authentication configurable +- PR#1168: remove shebang in context module +- PR#1045: cli: [free-task] raise error when no task-id specified +- PR#1056: Print warning to stderr +- PR#1057: raise error for non-existing task in list_task_output +- PR#1061: hub: [getRPMDeps] add strict behavior +- PR#1065: fix wrong message +- PR#1081: hub: [getPackageID] add strict behavior +- PR#1099: hub: [hasPerm] add strict behavior +- PR#732: koji.next.md: drop RHEL 5 requirements +- PR#1156: hub: unlimited NameWidth for kojifiles Apache location +- PR#1154: docs: update cheetah template user guide link +- PR#1148: docs: use "postgresql-setup initdb" to initialize database +- PR#1141: hub: document edit_tag argument types +- PR#1138: cli: fix "at least" typo in help text +- PR#1137: docs: unify "dnf" and "yum" instructions in server howto +- PR#1125: Ignore non-existing option when activate a session +- PR#1111: Don't retry if certificate is not readable +- PR#928: check tag existence in list-tagged cmd and listTagged* APIs +- PR#1127: only pass new incl_blocked call opt if it is explicitly needed +- PR#1124: tooltip for search field +- PR#1115: Do not require split_debuginfo +- PR#1123: fix wrong old value in postBuildStateChange callback +- PR#1097: hub: [getTaskInfo] add strict behavior +- PR#1098: cli: [download-task] readable error when no task found +- PR#1096: cli: fix typos in *-notification error msg +- PR#1072: Include WadersOS mention to 'koji runs here' doc +- PR#1094: hub: [postBuildStateChange] passing the newest build info +- PR#1066: Simple mode for mergerepos +- PR#1091: more informative error for invalid scm schemes +- PR#1003: update jenkins configuration +- PR#947: exclude py compiled files under util/ +- PR#965: check rpm headers support directly +- PR#978: get_next_release should check also running builds +- PR#1041: fix utf-8 output in CLI +- PR#1036: Add more test patterns for rpmdiff unit test. +- PR#1023: Expand user directory from config +- PR#1002: prioritize unittest2 +- PR#1000: Fix target handling in make_task +- PR#997: Fix rpmdiff's ignoring of size +- PR#1012: Fix isinstance with lists +- PR#1030: Create symlinks for builds imported onto non-default volumes +- PR#1021: Raise error for non-200 codes in download_file +- PR#1005: Add unit tests for check volume id substitution list +- PR#1027: [kojihub] add strict parameter in getBuildNotification +- PR#1016: raise Error when user not found in getBuildNotifications +- PR#1008: decode_args(): make a copy of the opts dict, rather than modifying it in-place +- PR#989: additional info on builders in channelinfo page +- PR#685: Rest of automated conversion from py3 changes +- PR#962: put source target scratch into policy_data in make_task +- PR#980: cli: rename _unique_path to unique_path, and deprecate the old one +- PR#900: enable batch multiCall in clone-tag +- PR#973: Check empty arches before spawning dist-repo +- PR#959: fix wrong tagNotification in tagBuildBypass API +- PR#969: Enable python3 on RHEL8 build +- PR#970: Add RISC-V (riscv64) to distrepo task +- PR#897: Fix use_host_resolv with new mock version (2017 Nov 22+) +- PR#868: allow force for pkglist_add +- PR#845: propagate exception correctly +- PR#831: Use unittest2 for rhel6 compatibility +- PR#873: Allow listing of blocked data in readTagGroups +- PR#940: Add --enabled --ready filters for list-channels +- PR#952: cli: [clone-tag] preserve build order +- PR#919: remove deprecated BuildRoot.scrub() +- PR#948: cli: don't show license for external RPM in rpminfo +- PR#879: cli: change bad reference in clone-tag +- PR#946: force using python2 to run script +- PR#925: Allow longer Build Target names + * Tue May 15 2018 Mike McLean - 1.16.0-1 - Fix CVE-2018-1002150 - distRepoMove missing access check - PR#884: Add option to configure DB port From 3fbdcac049b77c7ac3dc05df1d91ef1bbaaaebcc Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Mar 04 2019 21:56:58 +0000 Subject: [PATCH 2/9] also bump version in docs --- diff --git a/docs/source/conf.py b/docs/source/conf.py index 2803b05..871ec45 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -54,9 +54,9 @@ copyright = u'2017, Mike McLean, Mike B, Dennis Gilmore, Mathieu Bridon, Ian McL # built documents. # # The short X.Y version. -version = '1.16' +version = '1.17' # The full version, including alpha/beta/rc tags. -release = '1.16.0' +release = '1.17.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. From 40da6644d601caa69dd482e5965ab18608e03e3a Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Mar 04 2019 21:56:58 +0000 Subject: [PATCH 3/9] fix date --- diff --git a/koji.spec b/koji.spec index 91c11e6..bd82962 100644 --- a/koji.spec +++ b/koji.spec @@ -713,7 +713,7 @@ fi %endif %changelog -* Thu Feb 28 2018 Mike McLean - 1.17.0-1 +* Thu Feb 28 2019 Mike McLean - 1.17.0-1 - PR#1227: Added volume id as argument to livemedia and livecd tasks - PR#1070: consolidate access to rpm headers - PR#1274: cve-2018-1002161 From bc0f04dfd9f757d6e276ed686cd770439ecadf2b Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Mar 04 2019 21:56:58 +0000 Subject: [PATCH 4/9] migration doc for 1.17 --- diff --git a/docs/source/migrating_to_1.17.rst b/docs/source/migrating_to_1.17.rst new file mode 100644 index 0000000..66c26ab --- /dev/null +++ b/docs/source/migrating_to_1.17.rst @@ -0,0 +1,30 @@ +Migrating to Koji 1.17 +====================== + +.. + reStructured Text formatted + +You should consider the following changes when migrating to 1.17: + +DB Updates +---------- + +This release some minor schema changes + + * the ``tag_external_repos`` table has a new ``merge_mode`` column + * the ``build_target.name`` column is now a TEXT field rather than VARCHAR + +As in previous releases, we provide a migration script that updates the +database. + +:: + + # psql koji koji Date: Mar 04 2019 21:56:58 +0000 Subject: [PATCH 5/9] Release notes for 1.17.0 --- diff --git a/docs/source/release_notes.rst b/docs/source/release_notes.rst index 4c087f5..2afd02e 100644 --- a/docs/source/release_notes.rst +++ b/docs/source/release_notes.rst @@ -5,6 +5,7 @@ Release Notes .. toctree:: :maxdepth: 1 + release_notes_1.17 release_notes_1.16.2 release_notes_1.16.1 release_notes_1.16 diff --git a/docs/source/release_notes_1.17.rst b/docs/source/release_notes_1.17.rst new file mode 100644 index 0000000..106eb3e --- /dev/null +++ b/docs/source/release_notes_1.17.rst @@ -0,0 +1,243 @@ +Koji 1.17.0 Release notes +========================= + + +Migrating from Koji 1.16 +------------------------ + +For details on migrating see :doc:`migrating_to_1.17` + + + +Security Fixes +-------------- + +**CVE-2018-1002161 - SQL injection in multiple remote calls** + +| PR: https://pagure.io/koji/pull-request/1274 + +This release includes the fix for :doc:`CVE-2018-1002161` + + +Client Changes +-------------- + +**Volume id option for livemedia and livecd tasks** + +| PR: https://pagure.io/koji/pull-request/1227 + +The ``spin-livecd`` and ``spin-livemedia`` commands now accept a ``--volid`` +argument to specify the volume id for the media. If unspecified, the +volume id is chosen via the same heuristic as before. + +Volume ids must be 32 characters or less. + + + +**Build order preserved by clone-tag** + +| PR: https://pagure.io/koji/pull-request/1014 + +This is an improvement to the ``clone-tag`` command. Previously, when the +command was used without the ``--latest-only`` option, it could get the +ordering of builds wrong in the destination tag. Now, the order will +match the source tag. + + + +**Configurable authentication timeout** + +| PR: https://pagure.io/koji/pull-request/1172 + +Previously, the network timeout during authentication was hard coded to +60 seconds. It is now configurable via the ``auth_timeout`` configuration +option. + + +**Additional information from list-channels command** + +| PR: https://pagure.io/koji/pull-request/940 + +The ``list-channels`` command now shows three separate host counts for +each channel: + +- the number of enabled hosts in the channel +- the number of ready hosts in the channel +- the number of disabled hosts in the channel + + +**The free-task command requires at least one task-id** + +| PR: https://pagure.io/koji/pull-request/1045 + +Previously this command was a no-op when given no arguments. Now it will return an +error. + + + +Library Changes +--------------- + +**Drop encode_int function** + +| PR: https://pagure.io/koji/pull-request/852 + +This is a follow up to the large integer support that we added in version 1.14 + +See also: :doc:`release_notes-1.14` + +The ``encode_int`` function is no longer used +and has been dropped from the library. + +Because we no longer call ``encode_int``, the hub will now always use i8 tags +when returning large integers, rather than returning them as strings in some +cases. + + +**Use custom Kerberos contxt with krb_login** + +| PR: https://pagure.io/koji/pull-request/1187 + +Clients can now pass in their own Kerberos context to +``ClientSession.krb_login()`` using +the ``ctx`` parameter. This is intended for multi-threaded clients. + + +**Custom keyboard interrupt handling in watch_tasks** + +| PR: https://pagure.io/koji/pull-request/981 + +The new ``ki_handler`` option for the ``koji_cli.lib.watch_tasks()`` function +allows other cli tools to set their own handler for keyboard interrupts. +If specified, the value should be callable and will be called when a +keyboard interrupt is encountered. +If unspecified, the original behavior is retained. + + +**_unique_path() -> unique_path** + +| PR: https://pagure.io/koji/pull-request/980 + +The ``_unique_path`` function is deprecated. It has been replaced +by ``unique_path``. + + +Web UI Changes +-------------- + +**Additional info on builders in channelinfo page** + +| PR: https://pagure.io/koji/pull-request/989 + +The channelinfo page now shows enabled/ready status for each host and a count +for each. + + + +Builder Changes +--------------- + +**Builder task_avail_delay check** + +| PR: https://pagure.io/koji/pull-request/1176 + +This delay works around a deficiency in task scheduling. The default +delay is 300 seconds and can be adjusted with the ``task_avail_delay`` +option to kojid. However, it is unlikely that admins will need to +adjust this setting. + +Despite the name, this does not introduce any new delay compared to the +old behavior. The setting controls how long a host will wait before taking +a task in a given channel-arch "bin" when that host has an available +capacity lower than the median for that bin. Previously, such hosts +could wait forever. + + + +System Changes +-------------- + + +**Python 3 Support** + +| PR: https://pagure.io/koji/pull-request/1117 +| PR: https://pagure.io/koji/pull-request/891 +| PR: https://pagure.io/koji/pull-request/921 +| PR: https://pagure.io/koji/pull-request/1184 +| PR: https://pagure.io/koji/pull-request/1019 +| PR: https://pagure.io/koji/pull-request/1203 +| PR: https://pagure.io/koji/pull-request/969 +| PR: https://pagure.io/koji/pull-request/685 + +Support for Python 3 has been extended to all components of Koji. Including: + +- Hub +- Builder +- Web UI +- Utils + + + +**No more messagebus plugin** + +| PR: https://pagure.io/koji/pull-request/1043 + +The messagebus plugin has been dropped. The protonmsg plugin is still +available. + + + +**Simple mode for mergerepos** + +| PR: https://pagure.io/koji/pull-request/1066 + +External repos now have a ``merge_mode`` option. Valid values are +either ``koji`` (the old way) or ``simple`` (a new alternative). This +option can be set with the ``--mode`` option to the ``add-external-repo`` +or ``edit-external-repo`` commands. + +When an external repo is merged with simple mode, a number of the complex +filters that Koji normally applies are skipped. This mode still honors +the block list from Koji and ignores duplicate NVRAs, but otherwise +it simply merges the repo in. + +Multiple merge modes cannot be combined in a single tag. If a tag +has two external repos with different modes, then the repo will +fail to generate. + + +**Avoid "unknown task" errors in Kojira** + +| PR: https://pagure.io/koji/pull-request/1175 + +This is a bug fix for a minor race condition in Kojira that could cause +errors in the log and redundant repo regens. + + + +**Full filename display for kojifiles directory indexes** + +| PR: https://pagure.io/koji/pull-request/1156 + +This is simply a change to the default httpd configuration for serving +/mnt/koji. It adds ``NameWidth=*`` to ``IndexOptions`` so that long filenames +are fully displayed. + + + +**Broader support for target/source/scratch tests in channel policy** + +| PR: https://pagure.io/koji/pull-request/962 + +It is now possible to write channel policy rules based on +build target, source, and scratch options for task types other +than ``build``. + + + +**Longer Build Target names** + +| PR: https://pagure.io/koji/pull-request/925 + +Build target names can now be up to 256 characters, the same length +restriction as for tag names. From 8b69003c5ebeac5dcb81587dcf5d27a87ff578fd Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Mar 04 2019 21:56:58 +0000 Subject: [PATCH 6/9] update changelog --- diff --git a/koji.spec b/koji.spec index bd82962..78cfd90 100644 --- a/koji.spec +++ b/koji.spec @@ -713,7 +713,13 @@ fi %endif %changelog -* Thu Feb 28 2019 Mike McLean - 1.17.0-1 +* Fri Mar 1 2019 Mike McLean - 1.17.0-1 +- PR#1290: downloadTaskOutput fix for py3 +- PR#1300: require correct mod_wsgi +- PR#1301: use greetings list from lib +- PR#1284: replace urrlib.quote with six.moves +- PR#1286: correctly escape license in web ui +- PR#1292: define _sortByKeyFuncNoneGreatest as staticmethod - PR#1227: Added volume id as argument to livemedia and livecd tasks - PR#1070: consolidate access to rpm headers - PR#1274: cve-2018-1002161 From e0567a152d23249ec4821e72fa98349a50a3253a Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Mar 04 2019 21:56:58 +0000 Subject: [PATCH 7/9] update for recent prs --- diff --git a/docs/source/release_notes_1.17.rst b/docs/source/release_notes_1.17.rst index 106eb3e..891505d 100644 --- a/docs/source/release_notes_1.17.rst +++ b/docs/source/release_notes_1.17.rst @@ -165,9 +165,8 @@ System Changes | PR: https://pagure.io/koji/pull-request/921 | PR: https://pagure.io/koji/pull-request/1184 | PR: https://pagure.io/koji/pull-request/1019 -| PR: https://pagure.io/koji/pull-request/1203 -| PR: https://pagure.io/koji/pull-request/969 | PR: https://pagure.io/koji/pull-request/685 +| ...and many fixes Support for Python 3 has been extended to all components of Koji. Including: diff --git a/koji.spec b/koji.spec index 78cfd90..4326660 100644 --- a/koji.spec +++ b/koji.spec @@ -713,7 +713,13 @@ fi %endif %changelog -* Fri Mar 1 2019 Mike McLean - 1.17.0-1 +* Mon Mar 4 2019 Mike McLean - 1.17.0-1 +- PR#1293: fix dict encoding in py3 +- PR#1309: Fix binary output in cli in py3 +- PR#1317: fix deps for utils/vm subpackages on py3 +- PR#1315: fix checksum validation in CG_Importer +- PR#1313: Fix encoding issues with base64 data +- PR#1307: python3-koji-hub requires python3-psycopg2 - PR#1290: downloadTaskOutput fix for py3 - PR#1300: require correct mod_wsgi - PR#1301: use greetings list from lib From c0c57c4ffbfbcff2f30be77581e417b3e2795a27 Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Mar 04 2019 21:56:58 +0000 Subject: [PATCH 8/9] update changelog --- diff --git a/koji.spec b/koji.spec index 4326660..f06a5b0 100644 --- a/koji.spec +++ b/koji.spec @@ -713,7 +713,8 @@ fi %endif %changelog -* Mon Mar 4 2019 Mike McLean - 1.17.0-1 +* Tue Mar 5 2019 Mike McLean - 1.17.0-1 +- PR#1320: also remove nonprintable changelog chars in py3 - PR#1293: fix dict encoding in py3 - PR#1309: Fix binary output in cli in py3 - PR#1317: fix deps for utils/vm subpackages on py3 From f4e5798ecf2dee57d4be934b0beb70bb346e6f39 Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Mar 06 2019 14:18:24 +0000 Subject: [PATCH 9/9] update release date --- diff --git a/koji.spec b/koji.spec index f06a5b0..5a6a757 100644 --- a/koji.spec +++ b/koji.spec @@ -713,7 +713,7 @@ fi %endif %changelog -* Tue Mar 5 2019 Mike McLean - 1.17.0-1 +* Wed Mar 6 2019 Mike McLean - 1.17.0-1 - PR#1320: also remove nonprintable changelog chars in py3 - PR#1293: fix dict encoding in py3 - PR#1309: Fix binary output in cli in py3