fm-orchestrator

Clone
Source Code
GIT

#939 local module build ends up with wrong stream
Closed: Fixed 5 years ago by mprahl. Opened 6 years ago by merlinm.

Closed: Fixed
Reopen Issue

When using the latest module-build-service package (module-build-service-2.2.0-1.fc28.noarch) in combination with the fix for issue #935 (python2-rpkg-1.54-1.fc28.noarch), my local module build ended up in the wrong stream--'f28' instead of 'stable'.

avocado$ rpm -q module-build-service python2-rpkg
module-build-service-2.2.0-1.fc28.noarch
python2-rpkg-1.54-1.fc28.noarch
avocado$ git status
On branch stable
Your branch is up to date with 'origin/stable'.

nothing to commit, working tree clean
avocado$ fedpkg module-build-local
2018-05-11 12:34:07,957 - MainThread - moksha.hub - WARNING - Cannot find qpid python module. Make sure you have python-qpid installed.
2018-05-11 12:34:08,156 - MainThread - root - ERROR - There are multiple streams to choose from for module platform.
2018-05-11 12:34:08,157 - MainThread - root - ERROR - Use '-s module_name:module_stream' to choose the stream
avocado$ fedpkg --release f28 module-build-local
2018-05-11 12:34:15,244 - MainThread - moksha.hub - WARNING - Cannot find qpid python module. Make sure you have python-qpid installed.
2018-05-11 12:34:15,426 - MainThread - root - ERROR - There are multiple streams to choose from for module platform.
2018-05-11 12:34:15,427 - MainThread - root - ERROR - Use '-s module_name:module_stream' to choose the stream
avocado$ fedpkg --release f28 module-build-local --stream stable
2018-05-11 12:34:40,965 - MainThread - moksha.hub - WARNING - Cannot find qpid python module. Make sure you have python-qpid installed.
2018-05-11 12:34:41,211 - MainThread - root - ERROR - There are multiple streams to choose from for module platform.
2018-05-11 12:34:41,212 - MainThread - root - ERROR - Use '-s module_name:module_stream' to choose the stream
avocado$ fedpkg --release f28 module-build-local --stream stable -s platform:f28
2018-05-11 12:35:25,438 - MainThread - moksha.hub - WARNING - Cannot find qpid python module. Make sure you have python-qpid installed.             
2018-05-11 12:35:25,668 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): pdc.fedoraproject.org                    
2018-05-11 12:35:26,153 - MainThread - urllib3.connectionpool - DEBUG - https://pdc.fedoraproject.org:443 "GET /rest_api/v1/unreleasedvariants/?page_size=1&variant_id=platform&variant_version=f28&active=True HTTP/1.1" 200 538                                                                        
2018-05-11 12:35:26,362 - MainThread - urllib3.connectionpool - DEBUG - https://pdc.fedoraproject.org:443 "GET /rest_api/v1/unreleasedvariants/?page_size=1&variant_id=platform&variant_version=f28&active=True&page=4 HTTP/1.1" 200 579                                                                 
2018-05-11 12:35:26,570 - MainThread - urllib3.connectionpool - DEBUG - https://pdc.fedoraproject.org:443 "GET /rest_api/v1/unreleasedvariants/?page_size=-1&variant_id=platform&variant_version=f28&active=True&variant_release=4 HTTP/1.1" 200 474                                                     
2018-05-11 12:35:26,579 - MainThread - module_build_service - INFO - Starting resolving with following input modules: ['platform:f28:4:00000000']   
2018-05-11 12:35:26,581 - MainThread - module_build_service - DEBUG - Testing avocado:f28:20180511173525:0-20180511173525.src with combination: (<Solvable #2 platform:f28:4:00000000-4.x86_64>,)                                                                                                        
2018-05-11 12:35:26,582 - MainThread - module_build_service - DEBUG - Jobs:                                                                         
2018-05-11 12:35:26,585 - MainThread - module_build_service - DEBUG -   - favor platform:f28:4:00000000-4.x86_64                                    
2018-05-11 12:35:26,589 - MainThread - module_build_service - DEBUG -   - install avocado:f28:20180511173525:0-20180511173525.src                   
2018-05-11 12:35:26,590 - MainThread - module_build_service - DEBUG - Transaction:                                                                  
2018-05-11 12:35:26,593 - MainThread - module_build_service - DEBUG -   - avocado:f28:20180511173525:0-20180511173525.src                          
2018-05-11 12:35:26,594 - MainThread - module_build_service - DEBUG -   - platform:f28:4:00000000-4.x86_64                                         
2018-05-11 12:35:26,595 - MainThread - module_build_service - INFO - Resolving done, possible requires: {frozenset({'platform:f28:4:00000000:x86_64', 'avocado:f28:20180511173525:0:src'})}
...
2018-05-11 12:37:43,549 - PoolThread-twisted.internet.reactor-0 - module_build_service - DEBUG - Creating repository in /home/merlinm/modulebuild/builds/module-avocado-f28-20180511173525/results
...
2018-05-11 12:44:38,973 - MainThread - moksha.hub - INFO - MokshaHub reactor stopped
avocado$ dnf --disablerepo='*' --repofrompath=modbuild,/home/merlinm/modulebuild/builds/module-avocado-f28-20180511173525/results module list
Added modbuild repo from /home/merlinm/modulebuild/builds/module-avocado-f28-20180511173525/results
modbuild                                                                                                             7.5 MB/s | 7.6 kB     00:00   
Last metadata expiration check: 0:00:00 ago on Fri 11 May 2018 12:47:18 PM CDT.
modbuild
Name                                Stream                             Version                                    Profiles                          
avocado                             f28                                20180511173525                             default                           

Hint: [d]efault, [e]nabled, [i]nstalled, [l]ocked
avocado$

It appears I can work around this issue by also specifying the desired stream for my module on the command line, but that shouldn't be necessary.

avocado$ fedpkg --release f28 module-build-local -s platform:f28 -s avocado:stable
...
2018-05-14 10:05:09,566 - MainThread - module_build_service - INFO - Starting resolving with following input modules: ['platform:f28:4:00000000']
2018-05-14 10:05:09,567 - MainThread - module_build_service - DEBUG - Testing avocado:stable:20180514150508:0-20180514150508.src with combination: (<Solvable #2 platform:f28:4:00000000-4.x86_64>,)
2018-05-14 10:05:09,567 - MainThread - module_build_service - DEBUG - Jobs:
2018-05-14 10:05:09,568 - MainThread - module_build_service - DEBUG -   - favor platform:f28:4:00000000-4.x86_64
2018-05-14 10:05:09,569 - MainThread - module_build_service - DEBUG -   - install avocado:stable:20180514150508:0-20180514150508.src
2018-05-14 10:05:09,569 - MainThread - module_build_service - DEBUG - Transaction:
2018-05-14 10:05:09,570 - MainThread - module_build_service - DEBUG -   - avocado:stable:20180514150508:0-20180514150508.src
2018-05-14 10:05:09,571 - MainThread - module_build_service - DEBUG -   - platform:f28:4:00000000-4.x86_64
2018-05-14 10:05:09,571 - MainThread - module_build_service - INFO - Resolving done, possible requires: {frozenset({'avocado:stable:20180514150508:0:src', 'platform:f28:4:00000000:x86_64'})}
...
2018-05-14 10:07:29,525 - PoolThread-twisted.internet.reactor-0 - module_build_service - DEBUG - Creating repository in /home/merlinm/modulebuild/builds/module-avocado-stable-20180514150508/results
...
avocado$ dnf --disablerepo='*' --repofrompath=modbuild,/home/merlinm/modulebuild/builds/module-avocado-stable-20180514150508/results module list
Added modbuild repo from /home/merlinm/modulebuild/builds/module-avocado-stable-20180514150508/results
Last metadata expiration check: 0:08:46 ago on Mon 14 May 2018 10:25:31 AM CDT.
modbuild
Name                           Stream                       Version                              Profiles                      
avocado                        stable                       20180514150508                       default                       

Hint: [d]efault, [e]nabled, [i]nstalled, [l]ocked
avocado$

Are you able to reproduce that with current MBS and rpkg? I've just tried reproducing that and it works as expected for me.

I'm still seeing the exact same problem on a freshly installed F28 VM with the latest MBS and rpkg. It still needs the work around for the module to end up in the correct stream.

avocado$ rpm -q module-build-service python2-rpkg rpkg-common
module-build-service-2.7.0-1.fc28.noarch
python2-rpkg-1.56-2.fc28.noarch
rpkg-common-1.56-2.fc28.noarch
avocado$ git remote -v
origin  https://src.fedoraproject.org/modules/avocado.git (fetch)
origin  https://src.fedoraproject.org/modules/avocado.git (push)
avocado$ git status
On branch stable
Your branch is up to date with 'origin/stable'.

nothing to commit, working tree clean
avocado$
avocado$ fedpkg --release f28 module-build-local --stream stable -s platform:f28
2018-10-18 09:48:06,381 - MainThread - urllib3.util.retry - DEBUG - Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
2018-10-18 09:48:06,599 - MainThread - moksha.hub - WARNING - Cannot find qpid python module. Make sure you have python-qpid installed.
:0: UserWarning: You do not have a working installation of the service_identity module: 'No module named 'service_identity''.  Please install it from <https://pypi.python.org/pypi/service_identity> and make sure all of its dependencies are satisfied.  Without the service_identity module, Twisted can perform only rudimentary TLS client hostname verification.  Many valid certificate/hostname mappings may be rejected.
2018-10-18 09:48:06,792 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): mbs.fedoraproject.org
2018-10-18 09:48:07,308 - MainThread - urllib3.connectionpool - DEBUG - https://mbs.fedoraproject.org:443 "GET /module-build-service/1/module-builds/?name=platform&stream=f28&state=ready&verbose=True&order_desc_by=version&page=1&per_page=10 HTTP/1.1" 200 2323
2018-10-18 09:48:07,309 - MainThread - module_build_service - INFO - Starting resolving with following input modules: ['platform:f28:4:00000000']
2018-10-18 09:48:07,310 - MainThread - module_build_service - DEBUG - Testing avocado:f28:20181018144806:0-20181018144806.src with combination: (<Solvable #2 platform:f28:4:00000000-4.x86_64>,)
2018-10-18 09:48:07,310 - MainThread - module_build_service - DEBUG - Jobs:
2018-10-18 09:48:07,311 - MainThread - module_build_service - DEBUG -   - favor platform:f28:4:00000000-4.x86_64
2018-10-18 09:48:07,312 - MainThread - module_build_service - DEBUG -   - install avocado:f28:20181018144806:0-20181018144806.src
2018-10-18 09:48:07,312 - MainThread - module_build_service - DEBUG - Transaction:
2018-10-18 09:48:07,312 - MainThread - module_build_service - DEBUG -   - avocado:f28:20181018144806:0-20181018144806.src
2018-10-18 09:48:07,313 - MainThread - module_build_service - DEBUG -   - platform:f28:4:00000000-4.x86_64
2018-10-18 09:48:07,313 - MainThread - module_build_service - INFO - Resolving done, possible requires: {frozenset({'platform:f28:4:00000000:x86_64', 'avocado:f28:20181018144806:0:src'})}
...
2018-10-18 09:50:29,153 - PoolThread-twisted.internet.reactor-0 - module_build_service - DEBUG - Creating repository in /home/merlinm/modulebuild/builds/module-avocado-f28-2820181018144806/results
...
2018-10-18 09:57:35,284 - MainThread - moksha.hub - INFO - MokshaHub reactor stopped
avocado$
avocado$ dnf --disablerepo='*' --repofrompath=modbuild,/home/merlinm/modulebuild/builds/module-avocado-f28-2820181018144806/results module list
Added modbuild repo from /home/merlinm/modulebuild/builds/module-avocado-f28-2820181018144806/results
modbuild              0% [                    ] 3.7 MB/s | 7.7 kB     00:00 ETA
Last metadata expiration check: 0:00:00 ago on Thu 18 Oct 2018 10:03:50 AM CDT.
modbuild
Name            Stream         Version                  Profiles
avocado         f28            2820181018144806         default, minimal

Hint: [d]efault, [e]nabled, [i]nstalled, [l]ocked
avocado$
avocado$ fedpkg --release f28 module-build-local -s platform:f28 -s avocado:stable
2018-10-18 10:04:17,058 - MainThread - urllib3.util.retry - DEBUG - Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
2018-10-18 10:04:17,398 - MainThread - moksha.hub - WARNING - Cannot find qpid python module. Make sure you have python-qpid installed.
:0: UserWarning: You do not have a working installation of the service_identity module: 'No module named 'service_identity''.  Please install it from <https://pypi.python.org/pypi/service_identity> and make sure all of its dependencies are satisfied.  Without the service_identity module, Twisted can perform only rudimentary TLS client hostname verification.  Many valid certificate/hostname mappings may be rejected.
2018-10-18 10:04:17,605 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): mbs.fedoraproject.org
2018-10-18 10:04:18,367 - MainThread - urllib3.connectionpool - DEBUG - https://mbs.fedoraproject.org:443 "GET /module-build-service/1/module-builds/?name=platform&stream=f28&state=ready&verbose=True&order_desc_by=version&page=1&per_page=10 HTTP/1.1" 200 2323
2018-10-18 10:04:18,372 - MainThread - module_build_service - INFO - Starting resolving with following input modules: ['platform:f28:4:00000000']
2018-10-18 10:04:18,373 - MainThread - module_build_service - DEBUG - Testing avocado:stable:20181018150417:0-20181018150417.src with combination: (<Solvable #2 platform:f28:4:00000000-4.x86_64>,)
2018-10-18 10:04:18,373 - MainThread - module_build_service - DEBUG - Jobs:
2018-10-18 10:04:18,373 - MainThread - module_build_service - DEBUG -   - favor platform:f28:4:00000000-4.x86_64
2018-10-18 10:04:18,374 - MainThread - module_build_service - DEBUG -   - install avocado:stable:20181018150417:0-20181018150417.src
2018-10-18 10:04:18,374 - MainThread - module_build_service - DEBUG - Transaction:
2018-10-18 10:04:18,374 - MainThread - module_build_service - DEBUG -   - avocado:stable:20181018150417:0-20181018150417.src
2018-10-18 10:04:18,375 - MainThread - module_build_service - DEBUG -   - platform:f28:4:00000000-4.x86_64
2018-10-18 10:04:18,375 - MainThread - module_build_service - INFO - Resolving done, possible requires: {frozenset({'avocado:stable:20181018150417:0:src', 'platform:f28:4:00000000:x86_64'})}
...
2018-10-18 10:06:48,792 - PoolThread-twisted.internet.reactor-0 - module_build_service - DEBUG - Creating repository in /home/merlinm/modulebuild/builds/module-avocado-stable-2820181018150417/results
...
2018-10-18 10:13:13,521 - MainThread - moksha.hub - INFO - MokshaHub reactor stopped
avocado$
avocado$ dnf --disablerepo='*' --repofrompath=modbuild,/home/merlinm/modulebuild/builds/module-avocado-stable-2820181018150417/results module list
Added modbuild repo from /home/merlinm/modulebuild/builds/module-avocado-stable-2820181018150417/results
modbuild              0% [                    ] 7.6 MB/s | 7.8 kB     00:00 ETA
Last metadata expiration check: 0:00:00 ago on Thu 18 Oct 2018 10:25:21 AM CDT.
modbuild
Name            Stream         Version                  Profiles
avocado         stable         2820181018150417         default, minimal

Hint: [d]efault, [e]nabled, [i]nstalled, [l]ocked
avocado$
avocado$

Using the latest version of fm-orchestrator with the --offline option, the -s module:stream option no longer works around the issue--the module still ends up with platform's stream. Even putting stream: stream in the modulemd doesn't help.

This should fix it: https://pagure.io/fm-orchestrator/pull-request/1242.

I've tested it using following command:

$ $ mbs-manager build_module_locally --file=/home/hanzz/offline-demo/testmodule.yaml --offline -r /etc/yum.repos.d/fedora.repo -s platform:el8 --stream foo

The resulting module was "testmodule:foo".

Metadata Update from @mprahl:
- Issue assigned to jkaluza
5 years ago

Metadata Update from @mprahl:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)
5 years ago

Log in to comment on this ticket.

Metadata
None
None
None
Powered by Pagure 5.14.1
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.