How to append or insert record in final hive table

SUSHIL Registered Posts: 22 ✭✭✭


I am using the excel file to input the data.

Based on the visual recipe, I am transforming the input excel file as per standard format and storing the output in hive table.

The question is I need to regularly need to change my input based on the business date and then upload. These needs to flow and do transformation as per steps provided in the recipe and then it need to insert value into hive final table without overwriting the existing output or drop the existing value in table.

Can you provide me the steps to do the configuration to achieve the above results.

I don't want to add multiple files in the input as I need to upload on regular basis with removing existing file ans upload the new file as per my business case


  • Manuel
    Manuel Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Dataiku DSS Adv Designer, Registered Posts: 193 ✭✭✭✭✭✭✭


    In the input/output settings of a recipe, you can tick the append box for the output dataset, see attached.

    I hope this helps.

    SUSHIL Registered Posts: 22 ✭✭✭

    Thanks for your help.

    If we use recipe with csv (hive compatiable) mode in output, the above option get enabled. So that i can able enable and run the flow successfully.

    The data gets append instead of overwrite

Setup Info
      Help me…