Detect and alert secret leaks on python recipes

Options
rdagumampan
rdagumampan Registered Posts: 2

Hi, Id like to ensure our code recipes follows secure coding practice by not putting secrets (API Keys, Passwords, Tokens) in the code recipe. Is there a way to do this from Dataiku? Assuming we don't use external git repository.

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,713 Neuron
    edited July 10
    Options

    Hi, there are no built-in features for this but you can search inside recipe code using the Dataiku Python API:

    import dataiku
    
    client_handle = dataiku.api_client()
    variables_to_search = ['var1', 'var2']
    project_handle = client_handle.get_project('some project key')
    python_recipes = [i for i in project_handle.list_recipes() if i['type'] in ['python']]
    for python_recipe in python_recipes:
      recipe_name = python_recipe['name']
      recipe_handle = project_handle.get_recipe(recipe_name)
      recipe_script = recipe_handle.get_settings().get_payload().lower()
      if recipe_script:
        for var in variables_to_search:
          if var.lower() in recipe_script:
            print(f'Found variable {var} in recipe {recipe_name}')
    

    It shouldn't be too hard to customise this code to search for secrets. The following Python packages should help:

    https://pypi.org/project/ggshield/

    https://pypi.org/project/whispers/

Setup Info
    Tags
      Help me…