Join us on July 16th as we explore real-world Reinforcement Learning Learn more

partitioning

Level 1
partitioning

If I have a flow where I have partitioned files, and output file is not partitioned since its small

Is that ok, 

when date changes on partitioned tables that are being consumed, is that going  to have any impact on output

0 Kudos
1 Reply
Dataiker
Dataiker

Hi Venu, 

When you use partitioning feature, in all recipes you need to configure (in input/output tab of the recipe) the partition dependencies. 
If your input is partitioned by day, the output is not partitioned, the partition dependency is going to be set to "All Available": it means it's going to take all your partitions from the input to compute the output.  

You can watch this video training to understand better partitioning in DSS: 
https://community.dataiku.com/t5/Online-Events/Partitioning-in-Dataiku-DSS-Watch-on-Demand/ba-p/6148

Mattsco