Hide password in a pyspark recipe | use user secreys
Hi,
I'm creating a pyspark recipe in which i'm making an external connnection to jdbc to write to postgres database. However to do that I have to expose the password. is there anyway in dataiku we can keep the password hidden or use existing jdbc connections value properties in the pyspark recipr
Answers
-
CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭
Hi, @shakti29
! Can you provide any further details on the thread to assist users in helping you find a solution (insert examples like DSS version etc.) Also, can you let us know if you’ve tried any fixes already?This should lead to a quicker response from the community. -
@CoreyS
Sure Corey. We're using Dss 7.0 version. My main use case is i'm creating a pyspark recipe and then by using jdbc connecting to the postgres database. In order to do that i have to mention username, password, url and other details. The thing i dont want to do is write the password directly in the recipe. To avoid this, I have tried following approaches1) Use user secrets - I tried doing this. but it failed when identifying the user. I was using the api client.get_auth_info where client was created using client = dataiku.api_client(). This dint work.
2) Using existing connection: for this i had also thought of using exisiting connection details saved in dataiku and then getting username and password from it. but it again failed while creating a client object.
3) using global variables : This also dint work and failed at this dataiku.get_custom_variables()["driver"].
Approach I'm currently using: encrypting the password , but this is not really secure as any person who can access the script can get the password by decrypting it.
Can you please let me know how i can solve my problem
-
Hi,
Approach #1 should work. Check out this example: https://doc.dataiku.com/dss/latest/security/user-secrets.html#python
Did you pass with_secrets = True as an argument to get_auth_info()?
Best,
Pat