Best way to build partitioned datasets in scenario
NN
Neuron, Registered, Neuron 2022, Neuron 2023 Posts: 145 Neuron
Hi All,
Suppose i have a flow like image1 below which has 3 partitioned datasets
and i wish to run the flow on a single partition for all 3 datasets using scenario (DSS 8.0.1)
do i have to create a single step for each dataset like shown in image2 below(which means 3 steps for 3 datasets) ?
Best Answer
-
Hi,
No, you can specify only the last partitioned dataset (weather_conditions_summerized) with the CURRENT_DAY variable.
But you need to modify the build mode to allow recursivity: "build required" dataset or "force rebuild".
"Build required" will only update the partition if we detect new files in the current day sub path folder of the 1st filesystem dataset.
Answers
-
Thank you @Mattsco
.. That Helps