Building Scenario's issue
Hi ,
Sometimes due to lack of resources few jobs are getting failed/aborted.
the flow has sequential steps A->B->C->D. and using scenario i did a force -rebuild dependencies of D as the datasets are partitioned, and we want to trigger the latest partition only
when a scenario fails in dataiku which has retry set to 2. will it run from beginning or break point?
suppose it fails at C, will it build from A or will it retry from C
apologies if it is addressed elsewhere. I was not able to find it.
Thank You
Answers
-
Miguel Angel Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 118 DataikerHi,
The retry feature will restart the build step from the beginning. In our case it means it will try the build again, starting at A. To avoid the full rebuild, use 'build required datasets' instead of using 'force rebuild' . This way it will only rebuild the datasets that were not built.
In spite of the above, the main point of interest here is jobs failing/aborting because of lack of resources. The root reason for why that occurs should be investigated. We would gladly help in the investigation. For that create a support ticket in https://support.dataiku.com