Community Conundrum 25: Feature Visualization is now live! Read More

S3 Connection Problems

Level 1
S3 Connection Problems

Hello,

I am having problems connecting to an S3 bucket.  I get the following error:  

Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: ###############; S3 Extended Request ID: ###############################################)

I have tried applying policy at both the IAM User and bucket level with no luck.

Here is the bucket policy:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::#############:user/ServiceAccount"
},
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::dataikubucket123321",
"arn:aws:s3:::dataikubucket123321/*"
]
}
]
}

I've also tried at the IAM level:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket",
"s3:PutObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::dataikubucket123321/*",
"arn:aws:s3:::dataikubucket123321"
]
}
]
}

I can successfully create and test the connection using IAM Access ID and Secret Key. However, when I go to Add Dataset, I can see the folder structure when I "Browse" and see files when I hit "List Files", but when I hit "Test" it gives me the access denied error. 

Per AWS best practice, Block All Public Access is turned ON. The bucket and its files are encrypted using AWS-KMS, and I have configured the connection in DDS with the key.

2 Replies
Dataiker
Dataiker

Hi Matt,

can you try the following list of policies ?

  • s3:PutObject
  • s3:GetObject
  • s3:ListBucketMultipartUploads
  • s3:AbortMultipartUpload
  • s3:ListBucket
  • s3:DeleteObject
  • s3:GetBucketLocation
  • s3:ListJobs
  • s3:CreateJob
  • s3:HeadBucket

Furthermore, you might want to add the following to "*" (not required but useful):

  • s3:ListAllMyBuckets

More info here.

Take care,

Omar
Architect @Dataiku

Level 1
Author

Thanks.  I have all the relevant policies included.  The solution was that I had to modify the IAM user policy to give access to the bucket encryption key.  I can now read and write to S3 successfully.