We're excited to announce that we're launching the second installment of Dataiku Product Days Register Now

Add options to the Filter recipe : "is not equal to", "in a list", "not in a list"

I've worked in the SAS enterprise guide. In their Process Flow, when we filter the dataset there are almost all options available there.

Sadly in Dataiku, only a few options are available. It does not have basic filter conditions like "IS NOT EQUAL TO", "IN A LIST", "NOT IN A LIST", etc.

Although we can do these things inside the "prepare" recipe. But still, I think keeping these options inside the "Filter" recipe can make Dataiku more user-friendly.

3 Comments
natejgardner
Level 6

I'd love to have a few more sophisticated filter options. Of course, writing raw SQL is always an option, but this gets complicated.

Not equal is already available as "is different from," but "is in" and "is not in" would be nice to have for categoricals. For numericals, it'd be great if "is between" and "is not between" were options.

For the DSS engine, it'd be great if deep filtering were available for json objects and arrays.

Also, it'd be great if the visual filter builder were able to support more sophisticated logic. Currently, all the filtered values need to be combined with either and or or, but in some cases, users might want to use nested logic without needing to write an expression using formula language or SQL. This UI handles nested logic pretty well. It'd be awesome if logic like this could be expressed visually, I think it'd speed up development quite a bit and also help users intuitively understand what they're filtering for.

(
  (Condition1 OR Condition2) 
  AND 
  (Condition3 OR Condition4)
)
OR
(
  (Condition5 AND Condition6) 
  OR 
  (Condition7 AND Condition8)
)

 

PANKAJ
Level 3
Level 3

@natejgardner 

I completely agree with you.

There should be an option for the filtered values to combine with either 'AND" or 'OR' in the visual recipe.

AshleyW
Dataiker
Dataiker
Status changed to: Needs Info

Hi @PANKAJ ,

'not equal' is available as an option in the filter recipe as 'is different from'. To filter something that's in a list, you may try using a join recipe.

Please edit your original idea submission to reflect this, and feel free to add any of the conditions that @natejgardner mentioned.

@natejgardner, adding the ability to write nested filter logic visually is a great idea. Can you post that as a separate idea? 

Best,

Ashley