Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Added on April 16, 2019 11:55PM
Likes: 0
Replies: 4
Some datasets supports append, some not. Note that this mode is only available on recipe with output dataset using an infrastructure allowing append (e.g. it is not possible with HDFS).
Or you can use partitioning and put new data every day into a new partition.
First you need to activate partitioning on the dataset. You need to choose between time based or discrete partitioning. If you choose time base partitioning, then you can partition by year, year+month, or year+month+day.
Or you can partition using any discrete value, for example you choose partitioning by City, or by Country.
Then when you have activated the partitioning on the dataset, the icon should change.
After that you need to enter into the parent recipe and define the partition range what you want to calculate: