accessing project variables in a custom python preparation processor
Hello there !
I'm currently working on a plugin to add a new preparation processor that allows us to compute "half-life" weights in a prepare recipe.
The weighting function requires two parameters: a "delay" column and a "half-life" parameter.
We'd like for this parameter to be a project variable so we can test multiple values of the parameter without having to manually change it in the preparation recipes after each iteration.
I can't find a way to import the project variable in the processor.py of the plugin component. Any ideas or simple examples I could use ?
Thanks in advance,
Pierre.
Best Answer
-
Hi,
You can access the global and project variables using the dss_variables dict within the processor.py file.For instance:
def process(row): return dss_variables["delay"]
Answers
-
Alright that works! Thanks !
-
Hey, ok I've tried it and it does work using DSS engine, but when I use Spark it throws the following error in the "error" column while the output data is all NaN :
<type 'exceptions.NameError'> : NameError("global name 'dss_variables' is not defined",). Traceback (most recent call last):¶ File "<string>", line 18, in process¶
Is there anything I'm missing ? I'm using Dataiku version 7.1.2 if that helps.
Cheers,
Pierre.