Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
i have to delete data in a data set based on some conditions.
I want to achieve this via a auto trigger scenario maybe once in a day.
Do you think this will be able to achieve in dataiku? Pls let me know
Hello,
While there is no scenario step that can clear data based on specific conditions explicitly, this might be possible depending on the type of conditions / consistency (stays the same) of the conditions.
For example, if there was a dataset in which everyday you want to remove rows that contain 'XYZ', you could run a series of steps that:
1. filters the existing dataset using a filter recipe, (build dataset step)
2. clears the initial existing dataset, (clear dataset step)
3. writes the output of filtered recipe back to initial recipe (python step)
Alternatively, you could do all of this in a python step, as well.
Please let me know if you have any questions.
Roy