Issue #92: Executing Functional tests that require Multiple Hosts - standard-test-roles

standard-test-roles

#92 Executing Functional tests that require Multiple Hosts

Opened Nov 28, 2017 by mrniranjan. Modified Apr 14, 2018

I am part of the SSSD QE and we are in the process of pushing downstream tests to upstream. All our downstream tests are written in pytest and beakerlib and require Multiple Systems (Minimum 2, Maximum 3).

The beakerlib framework tests currently run in downstream CI using restraint.
Pytest framework tests currently use Multihost plugin [1]. to connect to multiple hosts.

Below are the systems that we use:
1. Controller:
2. system-1
3.. system-2

Controller Node has pytest, pytest-multihost-plugin, ansible, restraint installed,
system1 and system2 has the appropriate packages(sssd,389-ds-base) that are to be tested and restraint running.

We currently have Ansible playbooks that provisions, runs tests and teardowns the systems. These ansible playbooks run from the controller node.

Currently the multihost/functional tests are part of the sssd git[3].

For provisioning we use linchpin[2], to provision hosts on Openstack/beaker.

If beaker/openstack is not an option for Fedora CI, We could come up with provisioning using libvirt using pre-defined Qcow2 images.

We would like to hook up our ansible playbooks (provisions, runs, and teardowns) .

https://pagure.io/python-pytest-multihost/
https://github.com/CentOS-PaaS-SIG/linchpin
https://pagure.io/SSSD/sssd/blob/master/f/src/tests/multihost/basic

Metadata Update from @astepano:
- Issue assigned to astepano

Nov 30, 2017

mrniranjan commented Dec 1, 2017

Expanding on the above. To execute the test below are the details:

Controller system. This is the system where we clone the actual test cases, Contains test framework Packages . Following packages should be installed on the controller system

pytest
python-pytest-multihost (available on Fedora)
Git clone https://pagure.io/SSSD/sssd/

Apart from the above, you need another VM Containing Latest fedora (27) , This system is generally called "System Under Test"(SUT). SUT contains below packages
1. sssd from updates-testing repo (or the repo containing latest build)
2. Contains FQDN as it's hostname (DNS is not a strict requirement, as long as the Host name is resolved from Controller, so FQDN of SUT should be added in SUT as well as Controller system)
3. ssh running
4. Specify root password as "redhat"

Steps to run the tests:
1. On the controller system,
2. Make sure the SUT hostname is added to /etc/hosts of controller system
3. Git clone https://pagure.io/SSSD/sssd/
4. cd sssd/src/tests/python
5. python setup.py install
6. change to sssd/src/tests/multihost/basic
7. edit mhc.yaml file and change and change the string "idm1.example.test" to the hostname of the SUT.
8. Run py.test using below command
py.test --multihost-config=mhc.yaml -s -v test_basic.py

astepano commented Dec 6, 2017

@mrniranjan I followed your instructions.

I talked to you, on IRC. There is a summary:
1. standard-test-roles runs VM at 2222 port. You can change it, but it will require to write personal inventory script.
2. Maybe extend pytest_multihost to support port specification?

Thanks.

mrniranjan commented Apr 10, 2018

@astepano i was told internally by sturivny@redhat.com that non-standard ports for ssh are no longer the case and default ports on ssh should work.

Can you confirm ?, If so we need to move ahead with next steps ?

mrniranjan commented Apr 10, 2018

i would really not like to patch pytest_multihost or workaround just to make it work at non standard ports.

mrniranjan commented Apr 14, 2018

@astepano any update ?