Submit your inspiring success story or innovative use case to the 2022 Dataiku Frontrunner Awards! ENTER YOUR SUBMISSION

Recursive Build doesn't stop at metrics

Solved!
Jason
Level 3
Recursive Build doesn't stop at metrics

I have a dataset for which I have created a derivative metrics dataset (click the dataset in the flow, then click +dataset -> internal -> metrics)  This dataset exists on the flow with a line extending from the original dataset, but there is no recipe on that edge.

Capture.PNG

I am then using the newly created metrics dataset to create some additional information through several recipes in a separate flow zone.  Later, when I try to run a scenario to build the data in this flow zone, I choose recursive rebuild mode.  This causes the system to propagate recursively through all the previous datasources, including through the metrics dataset, which then of course propagates all the way back through the data that created the data from which the metrics comes.  In my case this is overselecting the data I want to have built

All of this is as expected so far, except I do not want the recursion to propagate beyond the metrics dataset.  For regular datasets, I can fix this by going under settings->advanced and changing the radio button for "rebuild behavior" from "normal" to "explicit" and that will stop the propagation, but for this metrics dataset, there is no such radio button under settings->advanced.

Capture2.PNG

In Summary, I want to build the dataset at the red arrow, and propagate, but I need to stop propagation at the red circle.

Is there some kind of property I can add, or some setting somewhere I can change to fix this?  Otherwise, I'm looking at sequentially building each and every dataset individually in my scenario that builds this flow zone.

Thanks,
-Jason

0 Kudos
1 Solution
fchataigner2
Dataiker
Dataiker

Hi

if you want to "break" the recursion when DSS computes what to rebuild, you have to set the rebuild behaviour in the dataset's Settings > Advanced:

Screenshot 2022-06-24 at 05.56.23.png

In your particular case, you have to do it on the SQLServer dataset  from which metrics are computed, because metrics dataset are merely views, and aren't computed, so they don't have any setting related to building.

View solution in original post

0 Kudos
2 Replies
fchataigner2
Dataiker
Dataiker

Hi

if you want to "break" the recursion when DSS computes what to rebuild, you have to set the rebuild behaviour in the dataset's Settings > Advanced:

Screenshot 2022-06-24 at 05.56.23.png

In your particular case, you have to do it on the SQLServer dataset  from which metrics are computed, because metrics dataset are merely views, and aren't computed, so they don't have any setting related to building.

0 Kudos
Jason
Level 3
Author

So the issue I'm worried about with this solution is this:  The dataset from which these metrics come, is the starting point for a recursive build that is controlled through a scenario.  If I stop the recursion with the dataset, then I still have a situation where recursion will not propagate correctly.  Looks like I'll be manually building each node in the problem region (because for future me, it will be more clear what is going on).

EDIT:  I spoke too soon.  This is the correct answer... because a dataset that is set with an explicit build behavior will still correctly propagate upstream when you command it to build.  Thanks, this is a good fix.