Hide password in a pyspark recipe | use user secreys

shakti29 Registered Posts: 4 ✭✭✭✭


I'm creating a pyspark recipe in which i'm making an external connnection to jdbc to write to postgres database. However to do that I have to expose the password. is there anyway in dataiku we can keep the password hidden or use existing jdbc connections value properties in the pyspark recipr


  • CoreyS
    CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭

    Hi, @shakti29
    ! Can you provide any further details on the thread to assist users in helping you find a solution (insert examples like DSS version etc.) Also, can you let us know if you’ve tried any fixes already?This should lead to a quicker response from the community.

  • shakti29
    shakti29 Registered Posts: 4 ✭✭✭✭

    Sure Corey. We're using Dss 7.0 version. My main use case is i'm creating a pyspark recipe and then by using jdbc connecting to the postgres database. In order to do that i have to mention username, password, url and other details. The thing i dont want to do is write the password directly in the recipe. To avoid this, I have tried following approaches

    1) Use user secrets - I tried doing this. but it failed when identifying the user. I was using the api client.get_auth_info where client was created using client = dataiku.api_client(). This dint work.

    2) Using existing connection: for this i had also thought of using exisiting connection details saved in dataiku and then getting username and password from it. but it again failed while creating a client object.

    3) using global variables : This also dint work and failed at this dataiku.get_custom_variables()["driver"].

    Approach I'm currently using: encrypting the password , but this is not really secure as any person who can access the script can get the password by decrypting it.

    Can you please let me know how i can solve my problem

  • pmasiphelps
    pmasiphelps Dataiker, Dataiku DSS Core Designer, Registered Posts: 33 Dataiker


    Approach #1 should work. Check out this example: https://doc.dataiku.com/dss/latest/security/user-secrets.html#python

    Did you pass with_secrets = True as an argument to get_auth_info()?



Setup Info
      Help me…