#411 Non-utf-8 mock output will break fedora-review
Opened 3 years ago by churchyard. Modified 3 years ago

I've just got:

01-31 14:28 root         DEBUG    Mock command: mock, -r, voikko, --enablerepo=local, --no-cleanup-after, --no-clean, --resultdir=/home/churchyard/rpmbuild/FedoraReview/1919688-voikko-fi/results, --chroot, --, rpm -i voikko-fi-2.4-3.fc33.src.rpm; rpmbuild --nodeps -bp $(rpm --eval %_specdir)/voikko-fi.spec;chmod -R  go+r  $(rpm --eval %_builddir)/* || :
01-31 14:28 root         DEBUG    Exception down the road...
Traceback (most recent call last):
  File "/home/churchyard/rpmbuild/FedoraReview/src/FedoraReview/review_helper.py", line 236, in run
    self._do_run(outfile)
  File "/home/churchyard/rpmbuild/FedoraReview/src/FedoraReview/review_helper.py", line 226, in _do_run
    self._do_report(outfile)
  File "/home/churchyard/rpmbuild/FedoraReview/src/FedoraReview/review_helper.py", line 99, in _do_report
    self._run_checks(self.bug.spec_file, self.bug.srpm_file, outfile)
  File "/home/churchyard/rpmbuild/FedoraReview/src/FedoraReview/review_helper.py", line 117, in _run_checks
    self.checks.run_checks(output=output, writedown=not Settings.no_report)
  File "/home/churchyard/rpmbuild/FedoraReview/src/FedoraReview/checks.py", line 382, in run_checks
    run_check(name)
  File "/home/churchyard/rpmbuild/FedoraReview/src/FedoraReview/checks.py", line 357, in run_check
    check.run()
  File "/home/churchyard/rpmbuild/FedoraReview/src/FedoraReview/plugins/generic_build.py", line 419, in run
    errmsg = Mock.rpmbuild_bp(self.srpm)
  File "/home/churchyard/rpmbuild/FedoraReview/src/FedoraReview/mock.py", line 429, in rpmbuild_bp
    errmsg = self._run_cmd(cmd)
  File "/home/churchyard/rpmbuild/FedoraReview/src/FedoraReview/mock.py", line 231, in _run_cmd
    output, error = p.communicate()
  File "/usr/lib64/python3.9/subprocess.py", line 1117, in communicate
    stdout = self.stdout.read()
  File "/usr/lib64/python3.9/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 2115: invalid continuation byte
01-31 14:28 root         ERROR    Exception down the road... (logs in /home/churchyard/.cache/fedora-review.log)

The problem is non-utf-8 stuff from rpmbuild -bp:

$ mock -r voikko --enablerepo=local --no-cleanup-after --no-clean --resultdir=/home/churchyard/rpmbuild/FedoraReview/1919688-voikko-fi/results --chroot -- 'rpm -i voikko-fi-2.4-3.fc33.src.rpm; rpmbuild --nodeps -bp $(rpm --eval %_specdir)/voikko-fi.spec;chmod -R  go+r  $(rpm --eval %_builddir)/* || :'
INFO: mock.py version 2.9 starting (python version = 3.9.1, NVR = mock-2.9-1.fc33)...
Start(bootstrap): init plugins
INFO: selinux enabled
Finish(bootstrap): init plugins
Start: init plugins
INFO: selinux enabled
Finish: init plugins
INFO: Signal handler active
Start: run
Start(bootstrap): chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start(bootstrap): cleaning package manager metadata
Finish(bootstrap): cleaning package manager metadata
INFO: enabled HW Info plugin
Mock Version: 2.9
INFO: Mock Version: 2.9
Finish(bootstrap): chroot init
Start: chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start: cleaning package manager metadata
Finish: cleaning package manager metadata
INFO: enabled HW Info plugin
Mock Version: 2.9
INFO: Mock Version: 2.9
Finish: chroot init
INFO: Running in chroot: ['rpm -i voikko-fi-2.4-3.fc33.src.rpm; rpmbuild --nodeps -bp $(rpm --eval %_specdir)/voikko-fi.spec;chmod -R  go+r  $(rpm --eval %_builddir)/* || :']
Start: chroot ['rpm -i voikko-fi-2.4-3.fc33.src.rpm; rpmbuild --nodeps -bp $(rpm --eval %_specdir)/voikko-fi.spec;chmod -R  go+r  $(rpm --eval %_builddir)/* || :']
warning: user vpv does not exist - using root
warning: group vpv does not exist - using root
warning: user vpv does not exist - using root
warning: group vpv does not exist - using root
warning: user vpv does not exist - using root
warning: group vpv does not exist - using root
warning: user vpv does not exist - using root
warning: group vpv does not exist - using root
setting SOURCE_DATE_EPOCH=1611964800
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.UrOGkW
+ umask 022
+ cd /builddir/build/BUILD
+ /usr/lib/rpm/redhat/gpgverify --keyring=/builddir/build/SOURCES/gpgkey-AC5D65F10C8596D7E2DAE2633D309B604AE3942E.gpg --signature=/builddir/build/SOURCES/voikko-fi-2.4.tar.gz.asc --data=/builddir/build/SOURCES/voikko-fi-2.4.tar.gz
gpgv: Signature made Sat Dec 19 15:15:29 2020 CET
gpgv:                using RSA key AC5D65F10C8596D7E2DAE2633D309B604AE3942E
gpgv: Good signature from "Harri Pitk�nen <hatapitk@iki.fi>"
gpgv:                 aka "Harri Pitk�nen <harri@puimula.org>"
gpgv:                 aka "Harri Pitk�nen <harri.t.pitkanen@iki.fi>"
+ cd /builddir/build/BUILD
+ rm -rf voikko-fi-2.4
+ /usr/bin/gzip -dc /builddir/build/SOURCES/voikko-fi-2.4.tar.gz
+ /usr/bin/tar -xof -
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd voikko-fi-2.4
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ RPM_EC=0
++ jobs -p
+ exit 0
Finish: chroot ['rpm -i voikko-fi-2.4-3.fc33.src.rpm; rpmbuild --nodeps -bp $(rpm --eval %_specdir)/voikko-fi.spec;chmod -R  go+r  $(rpm --eval %_builddir)/* || :']

This should do:

         p = Popen(
-            script, stdout=PIPE, stderr=STDOUT, shell=True, universal_newlines=True
+            script, stdout=PIPE, stderr=STDOUT, shell=True, encoding="utf-8", errors="replace"
         )

However the universal_newlines=True approach is omnipresent in the code base. Should I replace all?

Log in to comment on this ticket.

Metadata