Programmatic creation of Snowflake - SQl type dataset

yjagger

hi,

I created a dataset using below Dataiku's Python API.

project.create_sql_table_dataset(dataset_name, type, connection, table, schema)

I provide type = "Snowflake" and give the table and schema details, which already exists.

The problem is that, this operation also drops the table in Snowflake and re-creates it.

I saw in the logs, and found out that this happens when there is a schema mismatch between dataset and table. This is from the logs

Table already exists but with an incompatible schema: Type mismatch for column 11 (WAFER_NUM) : 'VARCHAR' in dataset, 'BIGINT'(-5:NUMBER) in table

Can you suggest any operations using which it does not drops the table in Snowflake, and a way for dataset to be created by automatically inferring whatever the schema is in Snowflake ?

---

Operating system used: Windows

---

Operating system used: Windows

Zach

Hi @yjagger
,

Could you please provide the following information?

1. When exactly is the table being dropped? Is it dropped immediately when you run "create_sql_table_dataset()", or does it happen at a later point?
2. Please provide the full code that you're using to create the dataset.
3. Where are you creating the dataset? For example, are you running the code in a notebook, or in a Python code recipe?