want to pick up data with relative date range(past * days)
Hello:
during pre-filter step, i only want to get the new dataset with limited data. so it should be updated autolly on daily basis.
I tried below formula but it is not workable.
( diff(val(now()),val('work_order_created_date')) >= 10 )
Please help me to get the right methods.
Thank you veyr much.
Operating system used: Windows
Answers
-
Manuel Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 193 ✭✭✭✭✭✭✭Hi,
The challenge you describe is solved using the partitions functionality, which allows you to process a subset of data from a relative time range (see attached image).
My suggestions is that you start by completing this academy course on the subject, https://academy.dataiku.com/advanced-partitioning.
I hope this helps.
Best regards
-
actually,i can not find the function in my screenshot, how to active it? btw, if any formula solution you can help to solve this issue? thank you very much.
-
also i can not find the date range function in building page which showed in video as well.
-
Manuel Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 193 ✭✭✭✭✭✭✭
Hi,
That is because your dataset is not partitioned yet. Please complete the academy course that I indicated to first learn the concepts and understand how to use it.
Best regards
-
in this prefilter stage, if possible to get only recent 10 days data only as one recipe.