Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I just tried the Writing data to snowflake with the Automatic fast-write using a S3 connection that uses "Credentials STS with AssumeRole" and the job build logs contain the AWS credentials in plain text like this:
{
"context": {
"type": "JOB_ACTIVITY",
"authIdentifier": "xxxxxx,
"projectKey": "TESTSNOWFLAKEWRITE",
"jobId": "Build_DKU_TUTORIAL_BASICS_101orders_date_changed_filtered_filtered__NP__2023-08-17T11-27-15.507",
"activityId": "compute_DKU_TUTORIAL_BASICS_101orders_date_changed_filtered_filtered_NP",
"activityType": "recipe",
"recipeType": "sampling",
"recipeName": "compute_DKU_TUTORIAL_BASICS_101orders_date_changed_filtered_filtered"
},
"type": "SQL_QUERY",
"id": "VjHZ3ILWd4I5kenp",
"startTime": 1692271644872,
"sqlQuery": {
"connectionUsageId": "HCFdrtLewGZ86g2m",
"connection": "xxxxx",
"query": "COPY INTO \"SANDBOX\".\"RUBEN\".\"TESTSNOWFLAKEWRITE_DKU_TUTORIAL_BASICS_101ORDERS_DATE_CHANGED_FILTERED_FILTERED\" FROM 's3://xxxx/dataiku/dataiku-snowflake-tmp/tmp.PDpZXNdJqTYBu3Y7/' CREDENTIALS=(aws_key_id='AWS_KEY_ID' aws_secret_key='CLEARTEXTSECRETKEY' aws_token='CLEARTEXTTOKEN') FILE_FORMAT = ( TYPE=csv NULL_IF=() FIELD_DELIMITER='\\t' FIELD_OPTIONALLY_ENCLOSED_BY='\"' ESCAPE_UNENCLOSED_FIELD=NONE) FORCE=false"
}
}
Note the COPY .... CREDENTIALS(aws_key_id=xxxx aws_secret_key=yyyy aws_token=zzz).
Obviously I would prefer AWS credentials not been written to the logs (masking them would be best).
Is there any way to prevent this?
Operating system used: Amazon Linux 2
I am not sure if this works since I have been unable to find documentation for this option but it's worth a try. Administration => Settings => Security & Audit => Login (LDAP, SSO) & Security => Other Xecurity Settings => Turn on "Encrypt secret keys".
That option (Administration > Settings > Security > Other security settings > Miscellaneous > Encrypt secret keys) was already set on my DSS 12.1.2 so that does not solve the issue.
There is a mention in the 12.2.0 release ntoes that might be applicable to your issue:
https://doc.dataiku.com/dss/latest/release_notes/12.html#security
Fixed credentials appearing in the logs when using Cloud-to-database fast paths