Using Project Variable in Partitioned Recipe

jkonieczny

I would like to use a Project Variable to explicity state which partition of a recipe I would like to run. See the attached screenshot - I used the variable expansion syntax but DSS did not interpret it correctly. Is there a way to do this?

Alex_Combessie

Hi, Indeed the syntax for referencing variables in the partition screen is ${myVariable}. Have you checked that the variable is actually defined? Cheers, Alexandre

jkonieczny

Yes, I have checked that the Project Variable is defined - it appears in the Project Variables and in the Variables section of the Python recipe that I am trying to run. However, when I attempt to build the dataset, it appears that it is trying to build against the partition "${current_month}" rather than "2019-01", which is what the variable points to:

compute_partitioned_output (${current_month})

I also get the following error after launching the dataset build:


Error in Python process: At line 8: : No JSON object could be decoded

augustin

Hi,

Actually, you won't be able to use your project variable here. However it's possible to do it within a scenario, from the step "Build", you can select your partitioned dataset and then use you project variable to define your partitions.

Pavle

This is now possible in v11.

https://doc.dataiku.com/dss/latest/release_notes/9.0.html#variables-expansion-support-in-build