From 2b2c6edf275f7b037d07cfedb9a4084f2de5e4e4 Mon Sep 17 00:00:00 2001 From: Petr Šplíchal Date: Oct 22 2018 09:44:48 +0000 Subject: [PATCH 1/2] New attribute "result" --- diff --git a/result.fmf b/result.fmf new file mode 100644 index 0000000..9985a20 --- /dev/null +++ b/result.fmf @@ -0,0 +1,28 @@ +name: result +type: string +purpose: specify how test result should be interpreted + +motivation: + Even if a test fails it might makes sense to execute it to be able + to manually review the results (ignore test result) or ensure the + behaviour has not unexpectedly changed and the test is still failing + (expected fail). + +stories: + - As a developer I have a test, I know it's failing for now, yet I + want to have it for future reference. + - As a tester I want to regularly execute the test but temporarily + ignore test result until more investigation is done and the test + can be fixed properly. + +notes: + The following values should be supported: + - respect: test result is respected (fails when test failed) + - ignore: ignore the test result (test always passes) + - xfail: make sure the test fails (fails when test passed) + +examples: | + result: ignore + +notes: + The default value is: "result: respect". From 10bb7f66e3f580796f6d64025183a201c84ccb37 Mon Sep 17 00:00:00 2001 From: Petr Šplíchal Date: Oct 31 2018 09:46:19 +0000 Subject: [PATCH 2/2] Update description for expected fail --- diff --git a/result.fmf b/result.fmf index 9985a20..a4702c7 100644 --- a/result.fmf +++ b/result.fmf @@ -19,7 +19,7 @@ notes: The following values should be supported: - respect: test result is respected (fails when test failed) - ignore: ignore the test result (test always passes) - - xfail: make sure the test fails (fails when test passed) + - xfail: expected fail (pass when test fails, fail when test passes) examples: | result: ignore