Detect and alert secret leaks on python recipes

rdagumampan

Hi, Id like to ensure our code recipes follows secure coding practice by not putting secrets (API Keys, Passwords, Tokens) in the code recipe. Is there a way to do this from Dataiku? Assuming we don't use external git repository.

Turribeach

Hi, there are no built-in features for this but you can search inside recipe code using the Dataiku Python API:

import dataiku

client_handle = dataiku.api_client()
variables_to_search = ['var1', 'var2']
project_handle = client_handle.get_project('some project key')
python_recipes = [i for i in project_handle.list_recipes() if i['type'] in ['python']]
for python_recipe in python_recipes:
  recipe_name = python_recipe['name']
  recipe_handle = project_handle.get_recipe(recipe_name)
  recipe_script = recipe_handle.get_settings().get_payload().lower()
  if recipe_script:
    for var in variables_to_search:
      if var.lower() in recipe_script:
        print(f'Found variable {var} in recipe {recipe_name}')

It shouldn't be too hard to customise this code to search for secrets. The following Python packages should help:

https://pypi.org/project/ggshield/

https://pypi.org/project/whispers/