Announcing the winners & finalists of the Dataiku Frontrunner Awards 2021! Read their inspiring stories

Stale schema in DataIKU flow

Solved!
allan
Level 2
Stale schema in DataIKU flow

We have a SQL Server backing store that we use to store our data.

A simple SQL recipe created a table that we now need to update the schema on - we're making 2 text fields longer.

I switched the schema to be Manually Define, updated the length of the two fields and ran the flow. The table was created and populated correctly.

When I switch the schema back to Automatically Generate, the flow now has the previous lengths so I assume they are cached somewhere. I see them being used in the logs to create the table.

How can I tell DataIKU to update those cached values based on the data in the current table?

0 Kudos
1 Solution
Clément_Stenac
Dataiker
Dataiker

Hi,

The length is stored in the schema of the dataset. Please go to the "Schema" tab of the dataset settings, select your column, and in the right part of the screen, update the "max length"

View solution in original post

0 Kudos
2 Replies
Clément_Stenac
Dataiker
Dataiker

Hi,

The length is stored in the schema of the dataset. Please go to the "Schema" tab of the dataset settings, select your column, and in the right part of the screen, update the "max length"

View solution in original post

0 Kudos
allan
Level 2
Author

@Clément_Stenac 

Thanks, that worked - it does look like that doesn't propagate when we use "Propagate schemas across Flow from here". Is there a way to make that happen, or do we need to adjust the length in the schema for each table individually?

0 Kudos
A banner prompting to get Dataiku DSS