Discover all of the brand-new features and improvements to existing capabilities in the Dataiku 11.3 updateLET'S GO

Does force rebuilding datasets and dependencies in scenarios update source SQL datasets?

Solved!
stephensunny
Level 3
Does force rebuilding datasets and dependencies in scenarios update source SQL datasets?

If my source datasets are tables from a SQL database and I have a SQL query recipe creating a separate dataset called "raw" from those source tables, when I run a scenario to build the "raw" dataset and select force rebuild all datasets and dependencies, do my source tables from the SQL database get rebuilt too? To clarify, my scenario has not been configured to build the source SQL database tables. It has only been configured to build the final "raw" dataset that outputs from the SQL query recipie.


Operating system used: Windows 10

0 Kudos
1 Solution
RoyE
Dataiker

Hi!

If I am understanding your description, you have X datasets used to build your 'raw' dataset. (Many to one).

In the case that there are no upstream recipes to these input datasets, the answer is no. They will not be rebuilt. DSS will only retrieve the dataset from these SQL datasets.

In the case that there are upstream recipes to these input datasets, the answer is yes. Running a force rebuild will forcibly rebuild all upstream datasets.

If you are in the latter situation and do not want to rebuild the input datasets, please select the option to run this recipe only, and it will not build upstream.

View solution in original post

0 Kudos
2 Replies
RoyE
Dataiker

Hi!

If I am understanding your description, you have X datasets used to build your 'raw' dataset. (Many to one).

In the case that there are no upstream recipes to these input datasets, the answer is no. They will not be rebuilt. DSS will only retrieve the dataset from these SQL datasets.

In the case that there are upstream recipes to these input datasets, the answer is yes. Running a force rebuild will forcibly rebuild all upstream datasets.

If you are in the latter situation and do not want to rebuild the input datasets, please select the option to run this recipe only, and it will not build upstream.

0 Kudos
stephensunny
Level 3
Author

Thank you, much appreciated

0 Kudos