#15 convert API tests to functional tests which talk real HTTP
Merged by dcallagh. Opened by dcallagh.
dcallagh/greenwave functional-tests  into  master

Download 15.patch

This is the direction I would like to go with the tests. Instead of mocking out the ResultsDB and WaiverDB services, the tests will start up a copy of the real services and set up results and waivers in them, and then ask Greenwave for a decision.

In the diff, it looks like these test cases are all new, but it's actually just the exact same cases we already had but converted to call the testdatabuilder fixture instead of using mocks.

It makes the test setup a bit more complicated, but it has a number of advantages:

  • Tests don't need to know all the details of how exactly ResultsDB and WaiverDB respond to a particular request
  • Tests don't need to know the details of what requests Greenwave will send
  • We avoid bugs where our mock requests do not represent how the other services actually behave (see for example #4)
  • If ResultsDB or WaiverDB change behaviour in a way that affects Greenwave's answers, we will find out from these tests

When Jenkins runs the functional tests we would have it set up a test environment in OpenShift (with real ResultsDB + WaiverDB + Greenwave services) and then run these same test cases. That gives even more realism since we will be running the same containers in a production-like way. Similar to what WaiverDB already has in its Jenkinsfile.

This looks great.

:thumbsup:

Pull-Request has been merged by dcallagh

Very cool!

Metadata