Appending dataset with python data frame in scenario
Hi wrote this python code to store SQL dataset output in another SQL dataset using python only. "output_df" data frame is having 1 row but while appending into the data set it's now inserting and returning the message with "0 rows successfully written".
I also put result of "output_df" data frame which I want to append into the "BENCHMARK_MR" dataset.
Operating system used: Windows
Operating system used: Windows
Answers
-
Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 317 Dataiker
Hi @Savan_Hipara
,
That's odd, I haven't been able to reproduce the 0 rows successfully written message if my input dataset contains data.
Would you be able to add a print statement immediately before your attempt to write out the data?print(output_dataset.shape)
Another thing you could try is to call write_dataframe() on output_dataset directly:output_dataset.write_dataframe(output_df)
If you still see that no rows are written, can you grab the schema for both "output_df" and the output dataset?print(output_df.dtypes) client = dataiku.api_client() project = client.get_default_project() output_dataset = project.get_dataset('BENCHMARK_MR') print(output_dataset.get_schema())
Thank you,
Sarina