Hide password in a pyspark recipe | use user secreys

[shakti29]

Hi,

I'm creating a pyspark recipe in which i'm making an external connnection to jdbc to write to postgres database. However to do that I have to expose the password. is there anyway in dataiku we can keep the password hidden or use existing jdbc connections value properties in the pyspark recipr

[CoreyS]

Hi, @shakti29! Can you provide any further details on the thread to assist users in helping you find a solution (insert examples like DSS version etc.) Also, can you let us know if you’ve tried any fixes already?This should lead to a quicker response from the community.

[shakti29]

@CoreyS Sure Corey. We're using Dss 7.0 version. My main use case is i'm creating a pyspark recipe and then by using jdbc connecting to the postgres database. In order to do that i have to mention username, password, url and other details. The thing i dont want to do is write the password directly in the recipe. To avoid this, I have tried following approaches

1) Use user secrets - I tried doing this. but it failed when identifying the user. I was using the api client.get_auth_info where client was created using client = dataiku.api_client(). This dint work.

2) Using existing connection: for this i had also thought of using exisiting connection details saved in dataiku and then getting username and password from it. but it again failed while creating a client object.

3) using global variables : This also dint  work and failed at this dataiku.get_custom_variables()["driver"].

Approach I'm currently using: encrypting the password , but this is not really secure as any person who can access the script can get the password by decrypting it.

Can you please let me know how i can solve my problem

[pmasiphelps]

Hi,

Approach #1 should work. Check out this example: https://doc.dataiku.com/dss/latest/security/user-secrets.html#python

Did you pass with_secrets = True as an argument to get_auth_info()?

Best,

Pat