Stack Post-Filter with Project variable

Options
LeFragan
LeFragan Registered Posts: 2 ✭✭✭✭
edited July 16 in Using Dataiku

Hey,

I have this project variable :

{

"filter_condition" : "timestamp>='2019-03-13' && timestamp<='2019-03-13')"

}

I want to use this variable on a Stack Post filter.

I tried to stack post filter using ${filter_condition} but i got this error :

An invalid argument has been encountered : Incorrect formula: '${filter_condition}' : Missing number, string, identifier, regex, or parenthesized expression(Parsing error at offset 0), caused by: ParsingException: Missing number, string, identifier, regex, or parenthesized expression(Parsing error at offset 0)

I also tried with "${filter_condition}" filter but it doesn't seem like its doing the job cauz im getting rows of 2019-04-14.

Answers

  • VinceDS
    VinceDS Dataiker, Alpha Tester, Dataiku DSS Core Designer Posts: 45 Dataiker
    edited July 17
    Options

    Hi,

    I believe you need to change your variable syntax:

    {
    "filter_condition" : "'2019-03-13'"
    }

    And then in the recipe post-filter use a formula like:

    timestamp >= asDate(${filter_condition}, "yyyy-MM-dd")

    You can also use this inside a Prepare recipe to create a column with your date variable and compute time difference inside the Prepare, which will make your Stack post-filter easier.

    Hope this helps

  • LeFragan
    LeFragan Registered Posts: 2 ✭✭✭✭
    edited July 17
    Options

    Idk if i can really do that, because i may have multiple conditions to test for example

    (timestamp>='2019-03-13T07:44:10.000000Z' && timestamp<='2019-03-13T09:44:10.000000Z') || (timestamp>='2019-03-19T12:28:43.000000Z' && timestamp<='2019-03-19T14:38:43.000000Z')
  • VinceDS
    VinceDS Dataiker, Alpha Tester, Dataiku DSS Core Designer Posts: 45 Dataiker
    edited July 17
    Options

    To do that you may need to create multiple variables and use a syntax like:

    timestamp >= asDate(${date1}, "yyyy-MM-dd") ||timestamp <= asDate(${date2}, "yyyy-MM-dd").....

    In any case you cannot have a logical expression in your variable.

Setup Info
    Tags
      Help me…