Every file will then also be copied to /usr/local/bin/, not just the test scripts. It doesn't matter that much, but might be a bit awkward or unexpected to the test? However, it's documented to be that way, so not wrong (or a reason to block this PR).

Every file will then also be copied to /usr/local/bin/, not just the test scripts. It doesn't matter that much, but might be a bit awkward or unexpected to the test? However, it's documented to be that way, so not wrong (or a reason to block this PR).

Would /usr/local/share be better?

The role needs to make sure rsync is installed on the target before it can successfully use the ansible synchronize module.

Per the spec, the playbook must exit with a non-zero status if any test fails. There needs to be a check somewhere for that.

Good point will fix.

Good catch. Will fix.

Fixed for review points.

This isn't really a log file from any test, it just summarizes the status of all tests (PASS or FAIL). As this isn't removed below, and thus not just a temporary vehicle, how about naming it "summary.txt" or "results.txt"?

Would /usr/local/share be better?

Not sure - it might actually be more important to keep the relative path between test and data file like in the original dist-git, so that you can run them unmodified from there as well (for developing the tests).

Can we assume that the testbed is always ephemeral? E. g. for autopkgtest it's supported (and plausible) to run them on production machines, as long as the test doesn't declare breaks-testbed. If that's a use case, maybe the whole lot can be put into /var/tmp/tests/ instead, to avoid clobbering the target system's package manager territory? If that's not a use case, just keep it like it is, as it doesn't matter for ephemeral test beds.

"MUST place the main readable output of the test suite into a test.log file in the artifacts variable folder" is what the test specification says about test.log. Since there can be multiple tests in the test suite, I think combining a summary like this matches the intent of the spec. Does that make sense?

If that's a use case, maybe the whole lot can be put into /var/tmp/tests/ instead, to avoid clobbering the target system's package manager territory?

I'm not against this specifically ... but what makes /usr/local the system package manager's teritorry? Isn't /usr/local on Linux systems about non packaged software?

MUST place the main readable output of the test suite into a test.log file in the artifacts variable folder

Thanks for pointing out, so this is fine.

makes /usr/local the system package manager's teritorry?

Sorry, not package manager of course, but it's still a persistent location. As I said, this is a moot point if testbeds are always ephemeral, otherwise (running tests on "real" machines should be supported) my feeling is that /var/tmp/ might be more appropriate.

Fixed for @martinpitt review points. Using /var/tmp/tests and /var/tmp/artifacts

Great, thanks! Looks nice now.

This synchronize task also needs ssh_args: "-o UserKnownHostsFile=/dev/null" just like the earlier synchronize, else it will leave behind a stray entry in ~root/.ssh/known_hosts that may cause problems on future runs.

Thanks. Fixed.

Verified with several scripts of my own creation with atomic, container, and vagrant tests subjects. Works well. LGTM.