Create dataset from AWS S3 bucket

Options
Carl
Carl Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 19 ✭✭✭✭

Hi,

I'm trying to create a dataset from AWS S3 bucket but somewhere I get this error message :

"Could not list buckets: The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: 1QYN9PKG48DK4PSG; S3 Extended Request ID: gM5Sk5ejQ2Nt5QewA5AkSDPVU0+8ocGYe/itU6DvchHHXZcm6wHAD7G6N3rJPbfoSxM0CshYyYE=)"

For information :

I've followed instructions via link :

https://knowledge.dataiku.com/latest/courses/basics/create-dataset/concept-connection.html

My connections is already configured in my settings and test connections shows "OK"

Screenshot 2022-04-19 at 14.44.09.png

I've created a "dss" folder in my S3 bucket so the path is kind of:

My S3 bucket is p8west

The folder name is dss

Now when I go to my project flow and I try to create a new dataset from AWS S3 :

Screenshot 2022-04-19 at 14.52.02.png

Why it show this error message when in my connection settings it shows "OK"?

Tagged:

Answers

  • Ignacio_Toledo
    Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 411 Neuron
    Options

    Hi @Carl
    ,

    Have you checked the policies attached to the user with those security credentials?

    Maybe they are limited, and they don't allow you to list the contents of the buckets. For example, I'm using the most unsecure policy for one connection (because it is not sensitive data):

    Screenshot from 2022-04-20 08-31-41.png

    My suspicion, is that the security keys you provided are valid, but they don't have all the permissions you need.

    Hope this helps,

    Ignacio

  • Carl
    Carl Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 19 ✭✭✭✭
    Options

    S3FullAcess is granted to anyone with my access keys, that's what I don't understand.

  • Ignacio_Toledo
    Ignacio_Toledo Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 411 Neuron
    Options

    Hi @Carl
    ,

    I went too quickly with my previous suspicious. It looks like the string you have in the field "Access Key" is actually the "secret". This would explain why you get the message that the key pair is not recognized. Or else, there is another typo in either the access key or the secret.

    Hope this helps,

    I.

  • Carl
    Carl Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 19 ✭✭✭✭
    Options

    Hi Ignacio,

    thank you for your feedback. I did some changes and I could finally access the file on AWS S3.

    I've change the Compartiement strategy and create a new pair of keys which I used in the settings and finally worked out.

    Best regards

    Carl

Setup Info
    Tags
      Help me…