Add options to the Filter recipe : "is not equal to", "in a list", "not in a list"

PANKAJ

I've worked in the SAS enterprise guide. In their Process Flow, when we filter the dataset there are almost all options available there.

Sadly in Dataiku, only a few options are available. It does not have basic filter conditions like "IS NOT EQUAL TO", "IN A LIST", "NOT IN A LIST", etc.

Although we can do these things inside the "prepare" recipe. But still, I think keeping these options inside the "Filter" recipe can make Dataiku more user-friendly.

natejgardner

I'd love to have a few more sophisticated filter options. Of course, writing raw SQL is always an option, but this gets complicated.

Not equal is already available as "is different from," but "is in" and "is not in" would be nice to have for categoricals. For numericals, it'd be great if "is between" and "is not between" were options.

For the DSS engine, it'd be great if deep filtering were available for json objects and arrays.

Also, it'd be great if the visual filter builder were able to support more sophisticated logic. Currently, all the filtered values need to be combined with either and or or, but in some cases, users might want to use nested logic without needing to write an expression using formula language or SQL. This UI handles nested logic pretty well. It'd be awesome if logic like this could be expressed visually, I think it'd speed up development quite a bit and also help users intuitively understand what they're filtering for.

(
  (Condition1 OR Condition2) 
  AND 
  (Condition3 OR Condition4)
)
OR
(
  (Condition5 AND Condition6) 
  OR 
  (Condition7 AND Condition8)
)

PANKAJ

@natejgardner

I completely agree with you.

There should be an option for the filtered values to combine with either 'AND" or 'OR' in the visual recipe.

Ashley

Hi @PANKAJ
,

'not equal' is available as an option in the filter recipe as 'is different from'. To filter something that's in a list, you may try using a join recipe.

Please edit your original idea submission to reflect this, and feel free to add any of the conditions that @natejgardner
mentioned.

@natejgardner
, adding the ability to write nested filter logic visually is a great idea. Can you post that as a separate idea?

Best,

Ashley

apichery

Is in and Is not in operators are scheduled !
Is not equal to is already supported (it's named is different from)

ThomasW

Hi,

Is in and Is not in operators are available starting version 11.2.0.

You will find them under the names Is any of and Is none of.

Thanks for your feedback!

Best,

Thomas