Since you updated the API it would be nice to add some doc about it.

I like it! +1

I changed the YAML validation code completely but I had to create another class for policies in gating.yaml file. I have to think about this more, there is probably nicer solution.

I wonder if it would be simpler/nicer to provide a CLI tool which can validate the policies?

On the server side we might then only need to add some API like "list all valid subject types" and "list all valid decision contexts" and then CLI tool would use that info to perform its checks.

It might be nice if people can use it as part of their CI workflow, to help prevent merging an invalid gating.yaml file in the first place.

I wonder if it would be simpler/nicer to provide a CLI tool which can validate the policies?

Server already knows how to parse and validate YAML file. Having this functionality in a separate would mean to duplicate some code or separate the code for new package/subpackage.

Another problem is that the CLI package can get outdated.

On the server side we might then only need to add some API like "list all valid subject types" and "list all valid decision contexts" and then CLI tool would use that info to perform its checks.

I would prefer to validate even the attributes and tags in YAML.

It might be nice if people can use it as part of their CI workflow, to help prevent merging an invalid gating.yaml file in the first place.

A POST request can verify the YAML file. Maybe it would be nicer to return HTTP 400 if file is invalid.

I'll drop the UI (we can add it later if needed) and work on the new API endpoint which will probably used soon.

Need to write test and if possible use !Policy and not some new tag (I'm currently using !RemotePolicy to make the verification be able to differentiate between policies set in configuration and remotely).

Updated.

I tried that... and I if do this request:

curl -d "@example-gating.yaml" -X POST localhost:5005/api/v1.0/validate-gating-yaml

where example-gating.yaml is:

--- !Policy
id: "testing_test"
product_versions:
  - fedora-26
decision_context: bodhi_update_push_testing
blacklist: []
rules:
  - !PassingTestCaseRule {test_case_name: dist.depcheck}

I get an error:

{
  "message": "Missing !Policy tag"
}

Am I doing something wrong?

Works when using --data-binary instead -d.

Cool, +1