#426 STI tests marked as failures when they actually passed, *** buffer overflow detected ***
Closed by churchyard. Opened by churchyard.

I've noticed that for the last couple days in https://src.fedoraproject.org/rpms/pypy3.9/pull-request/38 https://src.fedoraproject.org/rpms/pypy/pull-request/44 https://src.fedoraproject.org/rpms/pypy3.9/pull-request/39 the STI tests succeed, with messages like this in the log:

Run test 'smoke': done. Test's exit code: 0

Yet the overall status of the test is reported as a failure. This happens both for Zuul and non-Zuul.

It seems like an infra error rather than failure, with:

TASK [include_role : str-common-final] *****************************************
TASK [str-common-final : Pull out the logs from test environment to test runner] ***
fatal: [sut]: FAILED! => {
    "changed": false,
    "cmd": "/usr/bin/rsync --delay-updates -F --compress --archive --rsh='/usr/bin/ssh -S none -i /CONFIG/id_rsa_artemis -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o UserKnownHostsFile=/dev/null' --out-format='<<CHANGED>>%i %n%L' root@3.22.208.39:/tmp/artifacts// /var/ARTIFACTS/work-tests.ymli46lu148/tests-sean99bi/",
    "rc": 12
}
MSG:
Warning: Permanently added '3.22.208.39' (ECDSA) to the list of known hosts.
*** buffer overflow detected ***: terminated
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.3]
PLAY RECAP *********************************************************************
sut                        : ok=34   changed=10   unreachable=0    failed=1    skipped=10   rescued=0    ignored=0   
---^---^---^---^---^---
# STDERR:
---v---v---v---v---v---
---^---^---^---^---^---

Hi,

Thanks for the report.

I see that the issue (seems related) has been reported recently for the rsync component: https://bugzilla.redhat.com/show_bug.cgi?id=2229654

  • Does it is only affect the rawhide branch ?
  • On which Zuul job it happen ?

Yes, this is related to Testing Farm, happens with STI tests.

Currently, most probably a bug in rsync, or somewhere near, but investigation is ongoing.

related to https://pagure.io/fedora-ci/general/issue/427

Contacted the maintainer @mruprich to take a look

Could you please pull https://bodhi.fedoraproject.org/updates/FEDORA-2023-563d5c4a26 tot he Fedora 39 nodes?

aha, blocked in tests, that is sad, well let's push that to stable

@adamwill this is stuck in bodhi, we should get it to F39, how does one do that?

There is an ongoing freeze; this will be pushed to stable after the freeze is over.

By the way, the f39 update was accepted for freeze exception - https://bugzilla.redhat.com/show_bug.cgi?id=2229654#c17. So now we just wait a bit and it will get there.

Adam informed me that now we just wait a bit and it will get there.

it's requested for stable push (by releng) now.

Thanks. I am aware of the freeze exception, I was suggesting installing the update on the CI nodes before it goes stable, to expedite the fix of this problem.

This update has been pushed to stable. 19 hours ago

Updating images ...

Works now, thanks.

Metadata Update from @churchyard:
- Issue status updated to: Closed (was: Open)

Metadata