Datasets created by Python API but unable to read in the recipe

Evaria

Hi,

I have created dataset using this code:

import dataiku


client = dataiku.api_client()

project = client.get_default_project()

params = {'connection': 'connection_name', 'mode': 'table', 'table': 'table_name', 'schema': 'schema_name'}
dataset = project.create_dataset(dataset_name, type='Snowflake', params=params)

I write to this dataset using python Dataiku API too by adding below when calling the dataset.

ignore_flow=True 

I run the code in a recipe and specify ds as the output.

Both process is successful.

Then, I use this dataset (ds) with ignore_flow=True as an input of another recipe, but I get this error when I run the recipe:

Input dataset is not ready (table does not exist)

Input dataset ds is not ready, caused by: CodedSQLException: in running compute_ds: SQL table does not exist: ds

Additional technical details: Code: ERR_JOB_INPUT_DATASET_NOT_READY_NO_TABLE, type: com.dataiku.dip.exceptions.SourceDatasetNotReadyException

To solve this issue I change the dataset settings by directly searching the table in the connection (created in the previous recipe). Then, I try to change the dataset settings automatically using Dataiku API python. But, it is still get same error as above. So, it only works if I change manually dataset settings through Dataiku API. Although the parameters of the dataset settings is same, why using dataiku API, the issue still persist? Any solutions to solve this issue? Thank you in advance!

yonghyun

api 노드에서 design 노드의 데이터셋에 접근하려면 원격으로 접속하여서 사용하여야합니다.


https://community.dataiku.com/discussion/7493/dss-api-designer-read-dataset-from-dss-flow-in-r-python-api-endpoint