Hi - when you're able to have a look at this, could you please also add permission to the "aws-qa" role to create EFS for nfs mounts?
"User: arn:aws:sts::125523088429:assumed-role/aws-qa/dbrouwer is not authorized to perform: elasticfilesystem:DescribeFileSystems on the specified resource."

This is also related to the permission problem for cloudfront distribution that @kevin was looking at in https://pagure.io/fedora-infrastructure/issue/11842

Alright. I think I have s3 perms set. You will need to name your buckets starting with "qa"

On EFS, you just need describe? Or you need to start/list/do snapshots? or ?

For S3 - I can now get into the "create bucket" dashboard, but I still can't actually create the bucket.
I logged in and out and tried again but same result.

Failed to create bucket
To create a bucket, the s3:CreateBucket permission is required.

View your permissions in the IAM console . Identity and Access Management in Amazon S3 
API response
Access Denied

Huh, ok, let me take a look and see if I typoed something...

ah, might have missed a * there. Can you try now?

All good! I've created an S3 bucket thanks very much.

For EFS - I need mostly all of the permissions I think.
I need to be able to create, delete, mount, tag, backup and describe the filesystems.

ok. Try now? You likely have to start the filesystem name with qa...

Hmm I tried to create qa-openqa-webserver-fs on vpc-0afefac8bae905972 and got this error
User: arn:aws:sts::125523088429:assumed-role/aws-qa/dbrouwer is not authorized to perform: elasticfilesystem:TagResource on the specified resource.

I tried again and took away all the tags but I got the same error.

Huh. What did it say the resource was?

Error message doesn't give any further info on the "specified resource"

I retried and fiddled with the settings but got the same error in the end.

I also got this warning but i don't think it's relevant.

We recommend enabling the EFS service-linked role using AWS IAM. Service-linked roles allow you to easily delegate permissions to AWS services and gain additional transparency into when they are used on your behalf. Learn more

Hum. I think I have the same perms copr does set... and I think they are able to do this.

@praiskup did you have to do anything odd to use EFS with copr?

@kevin thank you and @tflink both for your help sorting out the aws cli access tokens for me.

Can I please expand this ticket to request a new, separate VPC for openqa to use?

I think the root of these EFS problems is that I have created the openqa EC2 instances on the same VPC that copr is using and there can only be one EFS mount point per availability region - which the copr efs is already using. If we have a separate openqa VPC, fingers crossed the issue will go away.

Sure, we can do that.

What region do you need? any particular settings, just similar to the copr one?

The new VPC needs to be in the "us-east-1" region since I understand that the Elastic IP addresses are regional and that would allow us to keep the address associated with the new domain name that you just created for us.

For everything else, yes duplicating the existing VPC settings that copr is using would be great.

I've created vpc-0a95baa0d250e2c1c can you see if that one works for you?

The new vpc is great and will work, but even with a subnet from this new vpc, I am still unfortunately having AccessDenied problems with the EFS.

Could you please double check the EFS permissions?

Example of error from the console:
User: arn:aws:sts::125523088429:assumed-role/aws-qa/dbrouwer is not authorized to perform: elasticfilesystem:TagResource on the specified resource.

Example of error from cli:
This command:
aws efs create-mount-target --file-system-id arn:aws:elasticfilesystem:us-east-1:125523088429:file-system/fs-0c37827cad8542654 --subnet-id subnet-06988e04049653ea9

Generates this error:
An error occurred (AccessDeniedException) when calling the CreateMountTarget operation: User: arn:aws:iam::125523088429:user/qa is not authorized to perform: elasticfilesystem:CreateMountTarget on the specified resource

Another example from cli:
aws efs tag-resource --resource-id arn:aws:elasticfilesystem:us-east-1:125523088429:file-system/fs-0c37827cad8542654 --tags Key=FedoraGroup,Value=qa
An error occurred (AccessDeniedException) when calling the TagResource operation: User: arn:aws:iam::125523088429:user/qa is not authorized to perform: elasticfilesystem:TagResource on the specified resource

So, the policy now has:

"Resource": "arn:aws:elasticfilesystem:::file-system/qa*"

Is there a way you could conform the file-system name to start with qa?

Otherwise I am not sure how to restrict things to allow you, but not allow you to change other groups volumes...

Hm I wonder if we're stuck in a catch-22 here because I don't have permission to "tag" my filesystem, I can't give it a name that starts with "qa-" so I can't benefit from these permissions.

Could you give fs-042a5a9f6b1ad83c0 a name for me? like "qa-openqa-webserver-efs".

so, I could not find any way to name it. ;(

I was able to add the FedoraGroup qa tag to it tho... can you see if it will let you manage it now?

Sadly this wasn't enough to let me create mount targets. :cry:

I can see the tag that you added:

{
    "FileSystems": [
        {
            "OwnerId": "125523088429",
            "CreationToken": "vpc-0a95baa0d250e2c1c",
            "FileSystemId": "fs-042a5a9f6b1ad83c0",
            "FileSystemArn": "arn:aws:elasticfilesystem:us-east-1:125523088429:file-system/fs-042a5a9f6b1ad83c0",
            "CreationTime": "2024-04-09T23:12:44+00:00",
            "LifeCycleState": "available",
            "NumberOfMountTargets": 0,
            "SizeInBytes": {
                "Value": 6144,
                "Timestamp": "2024-04-12T16:59:32+00:00",
                "ValueInIA": 0,
                "ValueInStandard": 6144,
                "ValueInArchive": 0
            },
            "PerformanceMode": "generalPurpose",
            "Encrypted": false,
            "ThroughputMode": "bursting",
            "Tags": [
                {
                    "Key": "FedoraGroup",
                    "Value": "qa"
                }
            ],
            "FileSystemProtection": {
                "ReplicationOverwriteProtection": "ENABLED"
            }
        }
    ]
}

But I have the same error:
An error occurred (AccessDeniedException) when calling the CreateMountTarget operation: User: arn:aws:iam::125523088429:user/qa is not authorized to perform: elasticfilesystem:CreateMountTarget on the specified resource

Try again now?

Sorry this is so iterative. ;(

And :tada: it's working - thank you very much for your help with this.

Awesome. :) Shall we keep this open for a bit? or just close it and go with a new one for the next issue? ;)

One final thing before we close this ticket, could you please delete "fs-0c37827cad8542654" - this was the first fs I created but on the wrong (copr's) vpc. It would be nice to clean that up.

Then yeah makes sense to open new tickets for next issues :)

Deleted.

Thanks