Appending dataset with python data frame in scenario

Options
Savan_Hipara
Savan_Hipara Dataiku DSS Core Concepts, Registered Posts: 3 ✭✭✭

Hi wrote this python code to store SQL dataset output in another SQL dataset using python only. "output_df" data frame is having 1 row but while appending into the data set it's now inserting and returning the message with "0 rows successfully written".

error.png

I also put result of "output_df" data frame which I want to append into the "BENCHMARK_MR" dataset.

df.png


Operating system used: Windows


Operating system used: Windows

Tagged:

Answers

  • Sarina
    Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer Posts: 315 Dataiker
    edited July 17
    Options

    Hi @Savan_Hipara
    ,

    That's odd, I haven't been able to reproduce the 0 rows successfully written message if my input dataset contains data.

    Would you be able to add a print statement immediately before your attempt to write out the data?

    print(output_dataset.shape)


    Another thing you could try is to call write_dataframe() on output_dataset directly:

    output_dataset.write_dataframe(output_df)


    If you still see that no rows are written, can you grab the schema for both "output_df" and the output dataset?

    print(output_df.dtypes)
    
    client = dataiku.api_client()
    project = client.get_default_project()
    output_dataset = project.get_dataset('BENCHMARK_MR')
    print(output_dataset.get_schema())


    Thank you,
    Sarina

Setup Info
    Tags
      Help me…