Sync without dropping table

Options
khairulfathi
khairulfathi Registered Posts: 11 ✭✭✭✭

Hello,

I'm trying to push processed data to external table (Oracle) as part of data integration. Using Sync recipe, Dataiku will try to drop the table first before loading any data. This is not possible in my case where the Oracle ID I've given does not have DROP privilege. I've tried checking 'Append instead of overwrite' and using Free output schema under settings but both are not working.

Is there any configuration to bypass this or other recipes that can achieve the same outcome?

Thanks for your assistance.

Khairul

Best Answer

  • AlexandreV
    AlexandreV Dataiker Posts: 28 Dataiker
    Answer ✓
    Options

    Hello Khairul, sorry for the delay,

    The “Free output schema” should prevent modifications of the schema before the sync recipe to cause change of the schema of the output dataset. If you don’t have the rights to drop table it’s probably good to chose this option.

    The “Append instead of overwrite” option should not require a Drop Table of your table if the schema are the same in the input and output of the sync.

    Can you confirm that the schema of the two datasets (input and output of sync recipe) are equal?

    If that’s the case can you provide the logs of the sync job before the DROP TABLE statement log?

    ie few lines before:

    [INFO] [dku.dataset.sql] - DROP TABLE "PROJECT_KEY_Table_name"

    Alex

Answers

Setup Info
    Tags
      Help me…