This is related to the previous ticket and comment: https://pagure.io/fedora-infrastructure/issue/8958#comment-656531

So, we already set this up a while back with a bucket named 'fedora-ci-bucket'

Can you just use that one? or would you like a fedora-ci-osci-01 one?

Good day.

It seems the bucket is empty:

aws s3 ls s3://fedora-ci-bucket --recursive

aws s3 cp readme s3://fedora-ci-bucket/ 
upload: ./readme to s3://fedora-ci-bucket/readme        

I cannot find who exactly requested that bucket. Google search doesn't help

site:pagure.io 'fedora-ci-bucket'

If it is OK to use it sure, we can use it.

I tried to upload file with public read access. But it fails:

~ aws s3api put-object-acl --bucket fedora-ci-bucket --key readme --acl public-read
An error occurred (AccessDenied) when calling the PutObjectAcl operation: Access Denied

~ aws s3 rm s3://fedora-ci-bucket/readme
delete: s3://fedora-ci-bucket/readme

~ aws s3 cp readme s3://fedora-ci-bucket/ --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers 
upload failed: ./readme to s3://fedora-ci-bucket/readme An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

It seems we cannot put for public data.

Also, I am not very confident AWS user, how do we upload files to fedora-ci-bucket from fedora-ci resources? Is there way to do this without token? The point is: the token I have has power to modify all AWS resources, which is not safe.

Thank you for help.

I can add those perms to it. I thought it had s3* set on it for your role...

We could make another user/token with just s3 access if you like?

We could make another user/token with just s3 access if you like?

If this is the right way to upload files from ci-pipelines, than yes, could you please create a token with limited access to the s3 bucket. So we can use that token for managing s3 files.

I can add those perms to it. I thought it had s3* set on it for your role...
Please, if possible.

Thank you very much for helping, I appreciate you help.

Friendly ping :-) Thank you!

ok, sorry for the delay here. ;(

IN your homedirectory on batcave01 is: fedora-ci-s3.csv that has the info in it.

Please delete the file once you have copied it off and do keep the token as secure as you are able.

It should have perms to that s3 bucket.

Feel free to re-open if you run into problems or there's more to do.

@kevin, hello,

I checked the token. It works. However, ...
Could you please tweak permissions please, that we can use this token to store files with public access? Here is some logs:

➜ aws sts get-caller-identity --profile s3
{
    "UserId": "AIDAR2OOCKQWSX6Y2WMU2",
    "Account": "125523088429",
    "Arn": "arn:aws:iam::125523088429:user/fedora-ci-s3"
}

Next OK:

➜ export AWS_PROFILE=s3
➜ aws s3 cp readme s3://fedora-ci-bucket/
upload: ./readme to s3://fedora-ci-bucket/readme
➜ aws s3 ls s3://fedora-ci-bucket --recursive
2020-10-08 10:18:53         51 readme
➜ aws s3 rm s3://fedora-ci-bucket/readme

Next fails:

➜  aws s3api put-object-acl --bucket fedora-ci-bucket --key readme --acl public-read
An error occurred (AccessDenied) when calling the PutObjectAcl operation: Access Denied

➜  aws s3 cp readme s3://fedora-ci-bucket/ --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers 
upload failed: ./readme to s3://fedora-ci-bucket/readme An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

Could you please change config, that we could store files with RO for all?

Thank you!!!

@pingou hi,

I just checked, the token doesn't have permission to upload files to s3 for public access. I am not sure why you moved ticket to Fixed.

Sorry for the long delay here. ;(

Can you try again now? I think we were missing a * at the end of the policy, so it wasn't allowing things for anything but the bucket name/top level.

ok, we got it figured out on IRC.

Turns out there is a default set 'block all public access' checkbox. You have to uncheck this for that bucket in order for it to allow you to make things public. :)

I am not sure why you moved ticket to Fixed.

My apologies, I do not know how I did this, must have made a mistake somewhere