Accessing Plugin parameters in code

surajtr

Can you please tell me how to access and update plugin input variables in the code so that we can run it in the loop and append the ouputs ?

jfyuen

Hi,

That would be using the first code snippet, with something like that:

import dataiku
import pandas as pd

client = dataiku.api_client()
project = client.get_project("PROJECT_KEY_TO_CHANGE")
recipe = project.get_recipe("RECIPE_NAME_TO_CHANGE")
recipe_settings = recipe.get_settings()

dfs = []

out_dataset_name = recipe_settings.get_flat_output_refs()[0]  # Assuming the recipe has one dataset output
output_ds = dataiku.Dataset(out_dataset_name)

for new_surveyid in ["VALUE_TO_CHANGE1", "VALUE_TO_CHANGE2", ...]: # the values for surveyid
    recipe_params = recipe_settings.get_recipe_params()
    recipe_settings["customConfig"]["surveyid"] = new_surveyid  # Make sure the variable name in the plugin is "surveyid", change it if needed
    recipe_settings.save()
    recipe.run()
    dfs.append(output_ds.get_dataframe())

# Write the concatenated output    
output_ds.write_with_schema(pd.concat(dfs))  

The concatenated output would be written in the output of the recipe.

Make sure to adapt the above code snippet to your needs and make a few tries. You may also want to change the dataset output.

Best,

Jean-François

jfyuen

Hi,

By changing the plugin input variable, do you mean inputs parameters of a plugin recipe?

If so, you can modify the recipe using the API (example from a notebook):

client = dataiku.api_client()
project = client.get_project("PROJECT_KEY_TO_CHANGE")
recipe = project.get_recipe("RECIPE_NAME_TO_CHANGE")
recipe_settings = recipe.get_settings()
recipe_settings.raw_params["customConfig"]["VARIABLE_NAME_TO_CHANGE"] = "NEW_VALUE_TO_CHANGE"
recipe_settings.save()
recipe.run()

If you mean the input of the plugin parameters, you may try something like this:

client = dataiku.api_client()
plugin = client.get_plugin("PLUGIN_NAME_TO_CHANGE")
plugin_settings = plugin.get_settings()
plugin_settings.get_raw()["config"]  # Get plugin parameters
plugin_settings.get_raw()["config"]["PARAMETER_NAME_TO_CHANGE"] = "NEW_VALUE_TO_CHANGE"
plugin_settings.save()

You should adapt the above samples to your problem and environment. Be careful when changing those parameters however, as if other uses are using the recipe or the plugin concurrently they will have the updated parameters.

Hope this answers your question,

Jean-François

surajtr

Thanks for your response.

What i actually want is run the custom-recipes (which is present inside a plugin) in a loop where i can pass values to the surveyid (which is defined in recipe.json) and append the output to a dataframe.

surajtr

Hi @jfyuen

In a single project, we can have multiple instances of the recipe (which is part of the plugin). So how to identify what to put in here :

recipe = project.get_recipe("RECIPE_NAME_TO_CHANGE")

jfyuen

You could reuse the same snippet and loop over the recipes (like the values to change).

The outputs would then be for all recipes that you can stack in another recipe, or you can combine them into one with the same method by appending dataframes.

Or depending on your data, you could also merge them as a single input (with an added identifier if it does not exist already), and use a single recipe to generate the output.