Is there a way to avoid AWS credentials being leaked in the logs when doing Snowflake fast write?

ecerulm

I just tried the Writing data to snowflake with the Automatic fast-write using a S3 connection that uses "Credentials STS with AssumeRole" and the job build logs contain the AWS credentials in plain text like this:

{
"context": {
"type": "JOB_ACTIVITY",
"authIdentifier": "xxxxxx,
"projectKey": "TESTSNOWFLAKEWRITE",
"jobId": "Build_DKU_TUTORIAL_BASICS_101orders_date_changed_filtered_filtered__NP__2023-08-17T11-27-15.507",
"activityId": "compute_DKU_TUTORIAL_BASICS_101orders_date_changed_filtered_filtered_NP",
"activityType": "recipe",
"recipeType": "sampling",
"recipeName": "compute_DKU_TUTORIAL_BASICS_101orders_date_changed_filtered_filtered"
},
"type": "SQL_QUERY",
"id": "VjHZ3ILWd4I5kenp",
"startTime": 1692271644872,
"sqlQuery": {
"connectionUsageId": "HCFdrtLewGZ86g2m",
"connection": "xxxxx",
"query": "COPY INTO \"SANDBOX\".\"RUBEN\".\"TESTSNOWFLAKEWRITE_DKU_TUTORIAL_BASICS_101ORDERS_DATE_CHANGED_FILTERED_FILTERED\" FROM 's3://xxxx/dataiku/dataiku-snowflake-tmp/tmp.PDpZXNdJqTYBu3Y7/' CREDENTIALS=(aws_key_id='AWS_KEY_ID' aws_secret_key='CLEARTEXTSECRETKEY' aws_token='CLEARTEXTTOKEN') FILE_FORMAT = ( TYPE=csv NULL_IF=() FIELD_DELIMITER='\\t' FIELD_OPTIONALLY_ENCLOSED_BY='\"' ESCAPE_UNENCLOSED_FIELD=NONE) FORCE=false"
}
}

Note the COPY .... CREDENTIALS(aws_key_id=xxxx aws_secret_key=yyyy aws_token=zzz).
Obviously I would prefer AWS credentials not been written to the logs (masking them would be best).


Is there any way to prevent this?

---

Operating system used: Amazon Linux 2

Turribeach

I am not sure if this works since I have been unable to find documentation for this option but it's worth a try. Administration => Settings => Security & Audit => Login (LDAP, SSO) & Security => Other Xecurity Settings => Turn on "Encrypt secret keys".

ecerulm

That option (Administration > Settings > Security > Other security settings > Miscellaneous > Encrypt secret keys) was already set on my DSS 12.1.2 so that does not solve the issue.

Turribeach

There is a mention in the 12.2.0 release ntoes that might be applicable to your issue:

https://doc.dataiku.com/dss/latest/release_notes/12.html#security

• Fixed credentials appearing in the logs when using Cloud-to-database fast paths