Tracing Variables

nharbour Registered Posts: 2

Is there a feature to trace the use of variables in Dataiku DSS? It would be very useful to be able to automate a search through Projects to identify where it's been used, so we can evaluate the impact of any change to that variable.

Best Answer

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,728 Neuron
    edited July 17 Answer ✓

    Hi, there are no built-in features for this but you can search inside recipe code using the Python API:

    import dataiku
    client_handle = dataiku.api_client()
    variables_to_search = ['var1', 'var2']
    project_handle = client_handle.get_project('some project key')
    python_recipes = [i for i in project_handle.list_recipes() if i['type'] in ['python']]
    for python_recipe in python_recipes:
        recipe_name = python_recipe['name']
        recipe_handle = project_handle.get_recipe(recipe_name)
        recipe_script = recipe_handle.get_settings().get_payload().lower()
        if recipe_script:
            for var in variables_to_search:
                if var.lower() in recipe_script:
                    print(f'Found variable {var} in recipe {recipe_name}')

    This being a text search will likely lead to false positives depending on how unique your variables names are. The same code can be used to loop through all projects in your DSS instance but this may take a while to run depending on your instance size.

    A less advanced and more basic way will be to export the project, download the export, unzip it and then use Notepad++ Find in Files functionality to search for variables in all the project data files.


Setup Info
      Help me…