Custom recipe from plugin - read recipe configuration

Tomas
Tomas Registered, Neuron 2022 Posts: 121 ✭✭✭✭✭

Hi,

I have a custom plugin (recipe) in my project and I would like to access the recipe's configuration, specially the partition range:

image.png

In the code I have only access to these functions:

from dataiku.customrecipe import (
get_input_names_for_role,
get_output_names_for_role,
get_recipe_config,
get_plugin_config,
)

How can I get the values 2020-09-08 / 2020-09-10 ?

Answers

  • NedM
    NedM Dataiker Posts: 8 Dataiker

    Hi Tomas,

    In partitioned code recipes, DSS exposes the partition identifiers of the input and output datasets via flow variables.

    The available flow variables vary depending on the partition dependency function, partitioning dimensions, and the name and number of input/output dataset.

    Therefore, the easiest way to find out which variables are available in a given code recipe, is to switch to the "variables" view on the recipe side-panel. Note that you may need to "validate" the code recipe for the variables to appear.

    p1.png

    To insert the variables into your code, you can simply click on a variable name in the side-panel; this will insert the code dataiku.dku_flow_variables[<variable name>] into your script (in the case of python).

    In the example shared above, I'm printing three of the available flow variables: the list of "Date" partitions from my source dataset, and the first and last "Date" partition of the source dataset. After running the recipe, the following is found in the logs:

    p2.png

    Finally, note that in the case of a plugin recipe, you may need to programmatically generate the names of certain flow varaibles e.g. the variables that include the name of an input or output dataset.

    Hopefully this helps!

    Ned

  • Tomas
    Tomas Registered, Neuron 2022 Posts: 121 ✭✭✭✭✭

    I resolved it differently, but thanks!

Setup Info
    Tags
      Help me…