is there a way to check table is not empty before running join recipe
Hi All,
i'm using Hive engine, Is there a way to check the joining table is not empty before running the join recipe?
scenario:
Table A left Join Table B.
Table A is empty. i'm getting below
com.dataiku.dip.exceptions.SourceDatasetNotReadyException: Input dataset <sampleproj.tablename> is not ready
my expectation is to, the recipe should run and return 0 records dataset. please help me if any such settings available or there is way to run the recipe without runtime exception.
Thanks in advance.
Regards, Kiran
Operating system used: windows 10
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptionsHi,
The SourceDatasetNotReadyException can't be turned off.
Instead, you can use metrics and check in a scenario to determine if any of the input tables are empty and fail the scenario and not rebuild the dataset. If you want to create an empty dataset in that case you can do so with a python step for example.
You can find an example here on how to use metrics and check to check if dataset is empty:
Reference Doc:
https://doc.dataiku.com/dss/latest/scenarios/checks.html#checks
Let us know if that helps