S3 Connection Problems

mattrosenthal
mattrosenthal Registered Posts: 3 ✭✭✭✭

Hello,

I am having problems connecting to an S3 bucket. I get the following error:

Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: ###############; S3 Extended Request ID: ###############################################)

I have tried applying policy at both the IAM User and bucket level with no luck.

Here is the bucket policy:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::#############:user/ServiceAccount"
},
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::dataikubucket123321",
"arn:aws:s3:::dataikubucket123321/*"
]
}
]
}

I've also tried at the IAM level:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket",
"s3:PutObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::dataikubucket123321/*",
"arn:aws:s3:::dataikubucket123321"
]
}
]
}

I can successfully create and test the connection using IAM Access ID and Secret Key. However, when I go to Add Dataset, I can see the folder structure when I "Browse" and see files when I hit "List Files", but when I hit "Test" it gives me the access denied error.

Per AWS best practice, Block All Public Access is turned ON. The bucket and its files are encrypted using AWS-KMS, and I have configured the connection in DDS with the key.

Best Answer

  • mattrosenthal
    mattrosenthal Registered Posts: 3 ✭✭✭✭
    Answer ✓

    Thanks. I have all the relevant policies included. The solution was that I had to modify the IAM user policy to give access to the bucket encryption key. I can now read and write to S3 successfully.

Answers

  • Omar
    Omar Dataiker Posts: 30 Dataiker

    Hi Matt,

    can you try the following list of policies ?

    • s3:PutObject
    • s3:GetObject
    • s3:ListBucketMultipartUploads
    • s3:AbortMultipartUpload
    • s3:ListBucket
    • s3:DeleteObject
    • s3:GetBucketLocation
    • s3:ListJobs
    • s3:CreateJob
    • s3:HeadBucket

    Furthermore, you might want to add the following to "*" (not required but useful):

    • s3:ListAllMyBuckets

    More info here.

    Take care,

    Omar
    Architect @Dataiku

Setup Info
    Tags
      Help me…