PR#345: Include logs in results.yml, use consistent prefix -

standard-test-roles

#345 Include logs in results.yml, use consistent prefix

Merged May 21, 2019 by astepano. Opened May 16, 2019 by psss.

psss/standard-test-roles logs into master

psss commented May 16, 2019

Include all log file names in the results.yml file.
Prefix all log files so that investigation is easier.

psss commented May 16, 2019

Example results.yml file:

results:
    - test: wizard/bad
      result: fail
      logs:
        - FAIL-wizard-bad.log
        - FAIL-wizard-bad-out.log
        - FAIL-wizard-bad-err.log
    - test: wizard/good
      result: pass
      logs:
        - PASS-wizard-good.log
        - PASS-wizard-good-out.log
        - PASS-wizard-good-err.log
    - test: wizard/problem
      result: error
      logs:
        - ERROR-wizard-problem.log
        - ERROR-wizard-problem-out.log
        - ERROR-wizard-problem-err.log

bgoncalv commented May 17, 2019

Thanks, it works for me and the YAML is valid, I'm just wondering if the indentation is the best one.

I'd suggest to do something like:

    test -f "$results" || echo 'results:' > "$results"
    printf '%s\n' \
        "  - test: $STR_BKR_TEST" \
        "    result: $result"      \
        "    logs:"  \
        >> "$results"
    for log in "$log_file_path" "$OUTPUTFILE" "$logfile_stdout" "$logfile_stderr"; do
        if [ -f "$log" ]; then
            local prefixed_log="$STR_ARTIFACTS_DIR/${status}-$(basename $log)"
            mv -f "$log" "$prefixed_log"
            echo "      - $(basename $prefixed_log)" >> "$results"
        fi
    done

so the results.yml would be shift to left by 2 white spaces (but it might need some change if we want extra line between each test case):

results:
  - test: shell/func
    result: pass
    logs:
      - PASS-shell-func.log
      - PASS-shell-func-out.log
      - PASS-shell-func-err.log
  - test: shell/login
    result: pass
    logs:
      - PASS-shell-login.log
      - PASS-shell-login-out.log
      - PASS-shell-login-err.log
  - test: shell/smoke
    result: pass
    logs:
      - PASS-shell-smoke.log
      - PASS-shell-smoke-out.log
      - PASS-shell-smoke-err.log

rebased onto 4fb1d4cd9e5fc842f5ac555dfd9f92260c30f865

May 20, 2019

psss commented May 20, 2019

Thanks for the review and feedback, @bgoncalv. Makes sense to get rid of the extra spaces. Pull request updated. This is how results.yml looks now for beakerlib:

results:
- test: wizard/bad
  result: fail
  logs:
  - FAIL-wizard-bad.log
  - FAIL-wizard-bad-out.log
  - FAIL-wizard-bad-err.log
- test: wizard/good
  result: pass
  logs:
  - PASS-wizard-good.log
  - PASS-wizard-good-out.log
  - PASS-wizard-good-err.log
- test: wizard/problem
  result: error
  logs:
  - ERROR-wizard-problem.log
  - ERROR-wizard-problem-out.log
  - ERROR-wizard-problem-err.log

and for the basic role:

results:
- test: good
  result: pass
  logs:
  - PASS-str_good.log
  - PASS-str_good-err.log
- test: bad
  result: fail
  logs:
  - FAIL-str_bad.log
  - FAIL-str_bad-err.log
- test: missing
  result: error
  logs:
  - ERROR-str_missing.log
  - ERROR-str_missing-err.log

I've added an extra line between test cases to make the output a bit more readable.

Edited May 21, 2019 by psss

bgoncalv commented May 21, 2019

thanks, it looks good to me

psss commented May 21, 2019

Thanks for quick review, @bgoncalv.

astepano commented May 21, 2019

Looks good.

astepano commented May 21, 2019

Commit dca4d6a4 fixes this pull-request

Pull-Request has been merged by astepano

May 21, 2019