is there a way to check table is not empty before running join recipe

korupolu_kiran
Level 1
is there a way to check table is not empty before running join recipe

Hi All,

i'm using Hive engine, Is there a way to check the joining table is not empty before running the join recipe?

scenario:

Table A left Join Table B.

Table A is empty. i'm getting below 

com.dataiku.dip.exceptions.SourceDatasetNotReadyException: Input dataset <sampleproj.tablename> is not ready

my expectation is to, the recipe should run and return 0 records dataset.  please help me if any such settings available or there is way to run the recipe without runtime exception.

Thanks in advance.

Regards, Kiran


Operating system used: windows 10

0 Kudos
1 Reply
AlexT
Dataiker

Hi,

The SourceDatasetNotReadyException can't be turned off. 

Instead, you can use metrics and check in a scenario to determine if any of the input tables are empty and fail the scenario and not rebuild the dataset. If you want to create an empty dataset in that case you can do so with a python step for example. 

You can find an example here  on how to use metrics and check to check if dataset is empty: 

https://community.dataiku.com/t5/Using-Dataiku/How-to-set-up-a-check-that-the-data-set-is-empty-0-ro...

Reference Doc: 

https://doc.dataiku.com/dss/latest/scenarios/checks.html#checks

Let us know if that helps

0 Kudos