accessing project variables in a custom python preparation processor

rnorm
rnorm Registered Posts: 9 ✭✭✭✭

Hello there !

I'm currently working on a plugin to add a new preparation processor that allows us to compute "half-life" weights in a prepare recipe.

The weighting function requires two parameters: a "delay" column and a "half-life" parameter.

We'd like for this parameter to be a project variable so we can test multiple values of the parameter without having to manually change it in the preparation recipes after each iteration.

I can't find a way to import the project variable in the processor.py of the plugin component. Any ideas or simple examples I could use ?

Thanks in advance,

Pierre.

Best Answer

  • StanG
    StanG Dataiker, Registered Posts: 52 Dataiker
    edited July 17 Answer ✓

    Hi,
    You can access the global and project variables using the dss_variables dict within the processor.py file.

    For instance:

    def process(row):
        return dss_variables["delay"]

Answers

  • rnorm
    rnorm Registered Posts: 9 ✭✭✭✭

    Alright that works! Thanks !

  • rnorm
    rnorm Registered Posts: 9 ✭✭✭✭

    Hey, ok I've tried it and it does work using DSS engine, but when I use Spark it throws the following error in the "error" column while the output data is all NaN :

    <type 'exceptions.NameError'> : NameError("global name 'dss_variables' is not defined",). Traceback (most recent call last):¶ File "<string>", line 18, in process¶

    Is there anything I'm missing ? I'm using Dataiku version 7.1.2 if that helps.

    Cheers,

    Pierre.

Setup Info
    Tags
      Help me…