Sync without dropping table

khairulfathi

Hello,

I'm trying to push processed data to external table (Oracle) as part of data integration. Using Sync recipe, Dataiku will try to drop the table first before loading any data. This is not possible in my case where the Oracle ID I've given does not have DROP privilege. I've tried checking 'Append instead of overwrite' and using Free output schema under settings but both are not working.

Is there any configuration to bypass this or other recipes that can achieve the same outcome?

Thanks for your assistance.

Khairul

AlexandreV

Hello Khairul, sorry for the delay,

The “Free output schema” should prevent modifications of the schema before the sync recipe to cause change of the schema of the output dataset. If you don’t have the rights to drop table it’s probably good to chose this option.

The “Append instead of overwrite” option should not require a Drop Table of your table if the schema are the same in the input and output of the sync.

Can you confirm that the schema of the two datasets (input and output of sync recipe) are equal?

If that’s the case can you provide the logs of the sync job before the DROP TABLE statement log?

ie few lines before:

[INFO] [dku.dataset.sql] - DROP TABLE "PROJECT_KEY_Table_name"

Alex

khairulfathi

Thank you @AlexandreV
for pointing to the right direction. Turns out there's a typo in one of the target column name and the column order got switched. Once fixed, all working perfectly now.