Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hello,
I'm trying to push processed data to external table (Oracle) as part of data integration. Using Sync recipe, Dataiku will try to drop the table first before loading any data. This is not possible in my case where the Oracle ID I've given does not have DROP privilege. I've tried checking 'Append instead of overwrite' and using Free output schema under settings but both are not working.
Is there any configuration to bypass this or other recipes that can achieve the same outcome?
Thanks for your assistance.
Khairul
Hello Khairul, sorry for the delay,
The โFree output schemaโ should prevent modifications of the schema before the sync recipe to cause change of the schema of the output dataset. If you donโt have the rights to drop table itโs probably good to chose this option.
The โAppend instead of overwriteโ option should not require a Drop Table of your table if the schema are the same in the input and output of the sync.
Can you confirm that the schema of the two datasets (input and output of sync recipe) are equal?
If thatโs the case can you provide the logs of the sync job before the DROP TABLE statement log?
ie few lines before:
[INFO] [dku.dataset.sql] - DROP TABLE "PROJECT_KEY_Table_name"
Alex
Hello Khairul, sorry for the delay,
The โFree output schemaโ should prevent modifications of the schema before the sync recipe to cause change of the schema of the output dataset. If you donโt have the rights to drop table itโs probably good to chose this option.
The โAppend instead of overwriteโ option should not require a Drop Table of your table if the schema are the same in the input and output of the sync.
Can you confirm that the schema of the two datasets (input and output of sync recipe) are equal?
If thatโs the case can you provide the logs of the sync job before the DROP TABLE statement log?
ie few lines before:
[INFO] [dku.dataset.sql] - DROP TABLE "PROJECT_KEY_Table_name"
Alex
Thank you @AlexandreV for pointing to the right direction. Turns out there's a typo in one of the target column name and the column order got switched. Once fixed, all working perfectly now.