Passing Variable to a Filter Recipe
Hi, this seems pretty trivial but I can't seem to get it to work;
I have declared a variable called "COBDate" = "2022-01-28"
I can also select this variable within the filter recipe and DSS reports the correct value within the variables preview, however when executing a simple formula based filter such as;
COBDate == variables["COBDate"]
results in no records
however running
COBDate == "2022-01-28"
does work, I'm not sure how the filter recipe is parsing my variable through, any suggestions
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,215 Dataiker
Hi @HeteshPatel
,Could you try changing the name of your variable to be different from the name of the column?
E.g define your variable as COBDate_var in your project variables and then use COBDate == variables["COBDate_var"]
Thanks,
-
Thanks for the response @Alex
I have tried that but no affect unfortunately.As a workaround for now I've resorted in using Python to filter my dataframe using the below code which works, just not sure why the Filter recipe doesn't like it, I have a feeling it is to do with the quotations possibly but not sure how to wrap the variable within the filter recipe formula?
df.query('COBDate == "{}"'.format(dataiku.get_custom_variables()["COBDate_var"]))
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,215 Dataiker
Hi,
Glad you found a workaround. The filter with a variable used in formula for filter worked fine on my end can you confirm :
1) The exact JSON on how your variable is defined
2) The type of the COBDate column ( date, string?)
3) Your DSS version
Thanks,
Alex
-
Hi Alex
1. Variable below;
{ "COBDate_var": "2022-01-31" }
2. Field COBDate is type String
3. DSS version is 9.0.5
Did some tests and it seems I can get it to work if I use the native filter criteria
but since I need to apply additional conditions which are OR conditions I switch to formula, DSS automatically evaluates the variable and hard codes it thus removing the dynamic nature of the variable.
strval('COBDate') == '2022-01-31'
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,215 Dataiker
Hi,
I had no issue using something COBDate == variables["COBDate_var"] within the formula of the filter recipes can you share an exact syntax you used within the formula filter?
-
Hey Alex,
I get null results when executing this, despite the query having records that match this criteria
( strval('LN_TypeOfSft') == 'ABC' ) && ( COBDate == variables["COBDate_var"] ) && (( strval('ParticipantName') == 'XYZ' ) || ( strval('ParticipantName') == 'KYC' ))
-
stephl Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 8 ✭
Hi, I have the same issue at my end.
May I know if you could successully solve that issue with the filter recipe?