You now have until September 15th to submit your use case or success story to the 2022 Dataiku Frontrunner Awards!ENTER YOUR SUBMISSION

Table overwritten instead of append due to schema inconsistency

Solved!
kuraigal_dss
Level 1
Table overwritten instead of append due to schema inconsistency

I'm trying to append data instead of overwrite with dataset including decimal field values, Standard in target DB is to define Decimal as column datatype. 

In this case, Dataiku by default defines datatype of this field as Float or Double in schema,  causing the table to be dropped and re-create due to schema inconsistency 

Is there any way to overcome this, I would like to retain the table and the data in it without having to create additional transient table

0 Kudos
1 Solution
fchataigner2
Dataiker
Dataiker

Hi,

DSS has no mapping for the DECIMAL type, so at the moment it will always consider the table schema as incompatible and attempt to recreate the table. Using a temporary table for flushing in the other table is a good solution.

View solution in original post

2 Replies
fchataigner2
Dataiker
Dataiker

Hi,

DSS has no mapping for the DECIMAL type, so at the moment it will always consider the table schema as incompatible and attempt to recreate the table. Using a temporary table for flushing in the other table is a good solution.

kuraigal_dss
Level 1
Author

Thanks, I tried the above approach and it worked.

Additionally I ended up using sql script instead of sql query recipe to load from the temporary table to final bypassing schema compatibility check 

0 Kudos

Setup info

?
Tags (1)
A banner prompting to get Dataiku