Run sql recipe with partition dataset
Hello,
I need to write a SQL recipe that build a dataset on a specific partition set as a project variable.
In my flow i have dozens of recipes that worked this way, i enter the partition ${execution_date} (close to the button run) and it works.
For SQL recipe when i give the argument 2022-01-20 the query succeed but when i give the project variable ${execution_date} it failed.
Is it a known problem on the sql recipe ?
Edit : The same error occured with python recipe
Thank you in advance
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,226 Dataiker
Hi @benarl
,I can confirm the same error "Illegal time partitioning value" error when expanding a variable in the partition dependency for an SQL recipe. It should work with most other recipe types.
As a workaround, you can use variable expansion in a scenario step to build the output dataset of the SQL recipe instead.
I will note this in our backlog.
Thanks,