since the DC move robosignatory seems to be failing to sign OSTrees when requested from the FCOS pipeline. Our logs show:
Sending ostree-sign request for build 32.20200611.20.0 Waiting for a response to the sent request Traceback (most recent call last): File "/usr/lib/coreos-assembler/cmd-sign", line 272, in <module> sys.exit(main()) File "/usr/lib/coreos-assembler/cmd-sign", line 42, in main args.func(args) File "/usr/lib/coreos-assembler/cmd-sign", line 88, in cmd_robosignatory robosign_ostree(args, s3) File "/usr/lib/coreos-assembler/cmd-sign", line 127, in robosign_ostree validate_response(response) File "/usr/lib/coreos-assembler/cmd-sign", line 267, in validate_response raise Exception(f"Signing failed: {response['failure-message']}") Exception: Signing failed: Could not download prod/streams/testing-devel/builds/tmp/ostree-commit-object: An error occurred (403) when calling the HeadObject operation: Forbidden script returned exit code 1
We need to make sure the credentials are configured properly. Here's where they are configured in ansible: https://pagure.io/fedora-infra/ansible/blob/master/f/roles/robosignatory/templates/robosignatory.toml.j2#_509
Basically, when using those credentials, can you aws s3 ls fcos-builds/prod/streams/testing-devel/builds/tmp/ostree-commit-object?
aws s3 ls fcos-builds/prod/streams/testing-devel/builds/tmp/ostree-commit-object
This was caused by the AWS policy being IP-limited to the PHX2 outbound IP. This is now updated to the IAD2 outbound IP, so should be fixed.
Metadata Update from @puiterwijk: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
thanks @puiterwijk