freeipa

#9571 Pytest 8 compatibility
Closed: fixed by abbra. Opened by abbra.

Closed: fixed
Reopen Issue

Fedora 41 plans to introduce Pytest 8: https://fedoraproject.org/wiki/Changes/Pytest_8

We need to test against Pytest 8 as it brings breaking changes. FreeIPA is not on the list of affected packages for some reason and is missing from the test COPR repository.

I moved my distro to Pytest 8.0 this February, haven't had any issues with FreeIPA CI related to that but I don't run the full tests suite.

Metadata Update from @ftrivino:
- Issue assigned to ftrivino

Metadata Update from @ftrivino:
- Custom field on_review adjusted to https://github.com/freeipa-pr-ci2/freeipa/pull/3584

Found one problem revealed with pytest 8.2.0+.

ipa-run-tests --log-cli-level=debug -vvra test_xmlrpc/test_stageuser_plugin.py::TestDuplicates --setup-show
============================= test session starts ==============================
platform linux -- Python 3.12.5, pytest-8.2.0, pluggy-1.5.0 -- /usr/bin/python3
cachedir: /.pytest_cache
rootdir: /usr/lib64/python3/site-packages/ipatests
plugins: multihost-3.4, sourceorder-0.6.0
collecting ...
...
=========================== short test summary info ============================
ERROR test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_active_same_as_staged - ipalib.errors.NotFound: tuser: user not found
==================== 4 passed, 1 warning, 1 error in 4.58s =====================

pytest-8.2.0.failure.log

With fixed https://github.com/pytest-dev/pytest/issues/12135
order of teardowns was changed.

Setup plan for pytest 8.1.2:

test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_active_same_as_preserved 
      SETUP    C xmlrpc_setup
      SETUP    C user4 (fixtures used: xmlrpc_setup)
      SETUP    C user5 (fixtures used: xmlrpc_setup)
        test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_active_same_as_preserved (fixtures used: request, user4, user5, xmlrpc_setup)
test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_staged_same_as_active 
      SETUP    C stageduser4 (fixtures used: xmlrpc_setup)
        test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_staged_same_as_active (fixtures used: request, stageduser4, user4, xmlrpc_setup)
test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_staged_same_as_preserved 
        test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_staged_same_as_preserved (fixtures used: request, stageduser4, user5, xmlrpc_setup)
test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_active_same_as_staged 
        test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_active_same_as_staged (fixtures used: request, stageduser4, user4, xmlrpc_setup)
      TEARDOWN C stageduser4
      TEARDOWN C user4
      TEARDOWN C user5
      TEARDOWN C xmlrpc_setup

setup plan for pytest 8.2.0:

test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_active_same_as_preserved 
      SETUP    C xmlrpc_setup
      SETUP    C user4 (fixtures used: xmlrpc_setup)
      SETUP    C user5 (fixtures used: xmlrpc_setup)
        test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_active_same_as_preserved (fixtures used: request, user4, user5, xmlrpc_setup)
test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_staged_same_as_active 
      SETUP    C stageduser4 (fixtures used: xmlrpc_setup)
        test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_staged_same_as_active (fixtures used: request, stageduser4, user4, xmlrpc_setup)
test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_staged_same_as_preserved 
        test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_staged_same_as_preserved (fixtures used: request, stageduser4, user5, xmlrpc_setup)
test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_active_same_as_staged 
        test_xmlrpc/test_stageuser_plugin.py::TestDuplicates::test_active_same_as_staged (fixtures used: request, stageduser4, user4, xmlrpc_setup)
      TEARDOWN C stageduser4
      TEARDOWN C user5
      TEARDOWN C user4
      TEARDOWN C xmlrpc_setup

With new pytest teardown of user5 fixture runs before user4 one.

Here is cleanup function (the raising one):

        def cleanup():                                                           
            existed = self.exists                                                
            try:                                                                 
                del_command()                                                    
            except errors.NotFound:                                              
                if existed:                                                      
                    raise                                                        
            self.exists = False

what is going on:
- test_staged_same_as_preserved test: user5 track is deleted (the state self.exists and actual user tuser).
- test_active_same_as_staged test: user4 track is created (the state self.exits and actual user tuser (same user name as user5 track!))
- teardown of user5 ( the cleanup finalizer) removes actual user tuser
- teardown of user4 ( the cleanup finalizer) can't remove nonexistent user tuser (already removed in previous step), but tracker thinks it must exist (because of state self.exists = True).
- failure

With the opposite order teardown of user4 removes tuser user, teardown of user5 can't remove nonexistent tuser user but it's expected because the state of the tracker is False.

master:

  • 18d550a3367710618675b87f0157685165bfe444 ipatests: make TestDuplicates teardowns order agnostic

ipa-4-12:

  • d102773ce24481c6797f71557b75e77921164285 ipatests: make TestDuplicates teardowns order agnostic

@slev
as the PR's commit message contains "Related: https://pagure.io/freeipa/issue/9571" and not "Fixes", I am not sure if I can close this ticket or if there are remaining issues. Are you aware of additional problems with pytest8?

@frenaud ,
I'm not aware of any existent problems related to Pytest 8. But I don't run full tests suite and can't say There is no more issues with it.

Let's close this ticket and open individual ones in case other issues would be discovered.

Metadata Update from @abbra:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)
Metadata
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.