Add options to the Filter recipe : "is not equal to", "in a list", "not in a list"

Options
PANKAJ
PANKAJ Partner, L2 Admin, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Registered Posts: 26 Partner

I've worked in the SAS enterprise guide. In their Process Flow, when we filter the dataset there are almost all options available there.

Sadly in Dataiku, only a few options are available. It does not have basic filter conditions like "IS NOT EQUAL TO", "IN A LIST", "NOT IN A LIST", etc.

Although we can do these things inside the "prepare" recipe. But still, I think keeping these options inside the "Filter" recipe can make Dataiku more user-friendly.

3
3 votes

Released · Last Updated

Comments

  • natejgardner
    natejgardner Neuron, Registered, Neuron 2022, Neuron 2023 Posts: 151 Neuron
    Options

    I'd love to have a few more sophisticated filter options. Of course, writing raw SQL is always an option, but this gets complicated.

    Not equal is already available as "is different from," but "is in" and "is not in" would be nice to have for categoricals. For numericals, it'd be great if "is between" and "is not between" were options.

    For the DSS engine, it'd be great if deep filtering were available for json objects and arrays.

    Also, it'd be great if the visual filter builder were able to support more sophisticated logic. Currently, all the filtered values need to be combined with either and or or, but in some cases, users might want to use nested logic without needing to write an expression using formula language or SQL. This UI handles nested logic pretty well. It'd be awesome if logic like this could be expressed visually, I think it'd speed up development quite a bit and also help users intuitively understand what they're filtering for.

    ((Condition1 OR Condition2)AND(Condition3 OR Condition4))OR((Condition5 AND Condition6)OR(Condition7 AND Condition8))

  • PANKAJ
    PANKAJ Partner, L2 Admin, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Registered Posts: 26 Partner
    Options

    @natejgardner

    I completely agree with you.

    There should be an option for the filtered values to combine with either 'AND" or 'OR' in the visual recipe.

  • AshleyW
    AshleyW Dataiker, Alpha Tester, Dataiku DSS Core Designer, Registered, Product Ideas Manager Posts: 161 Dataiker
    Options

    Hi @PANKAJ
    ,

    'not equal' is available as an option in the filter recipe as 'is different from'. To filter something that's in a list, you may try using a join recipe.

    Please edit your original idea submission to reflect this, and feel free to add any of the conditions that @natejgardner
    mentioned.

    @natejgardner
    , adding the ability to write nested filter logic visually is a great idea. Can you post that as a separate idea?

    Best,

    Ashley

  • apichery
    apichery Dataiker, Alpha Tester, Product Ideas Manager Posts: 62 Dataiker
    Options

    Is in and Is not in operators are scheduled !
    Is not equal to is already supported (it's named is different from)

  • ThomasW
    ThomasW Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered, Product Ideas Manager Posts: 5 Dataiker
    Options

    Hi,

    Is in and Is not in operators are available starting version 11.2.0.

    You will find them under the names Is any of and Is none of.

    Thanks for your feedback!

    Best,

    Thomas

Setup Info
    Tags
      Help me…