Failed to synchronize Hive Metastore for dataset error

jrod31
jrod31 Registered Posts: 14

Hi,

I am running 3 flows under the same project with the same prepare recipe and I get the following error message on the 3rd flow and not on the other 2 flow.

HIVE_METASTORE_SYNCHRONIZE occurred 1 times
Failed to synchronize Hive metastore for dataset

The source of the dataset are CSV files from a SharePoint.
The flow is setup as follow:

- The dataset is created based on several CSV files on a SharePoint
- A prepare recipe is added (a column rename recipe) and I get the error after running it.

I run it on Local Stream.
The other 2 flows run good without any errors.
I have seen other posts but can't seem to figure it out.

Any idea or thoughts on this issue?

Thanks,


Operating system used: Windows

Tagged:

Best Answer

  • Vamsi Ramakrishna
    Vamsi Ramakrishna Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 7 ✭✭✭✭
    Answer ✓

    Go to settings of the target dataset and uncheck the "Auto-sync to Hive metastore". In case if you want to create the dataset as table in the backend Hive DB, then make sure whatever the column names and datatypes in prepare recipe are inline with the output dataset schema as well as the Hive table structure which that dataset is loading.

    Thanks,

    Vamsi

Answers

Setup Info
    Tags
      Help me…