Data not writing to dataset in Python recipe

petertsui

I created a Python recipe that I get data from an API. I save the response to a panda dataframe, but when I use the DataSet.write_from_dataframe to write the dataframe result my output dataset. The rows is written, but the row is empty when I check the dnb_duns_detail_json dataset.

Operating system used: MacOS

Turribeach

As you screen shot shows Dataiku does see the row but doesn't show it so I suspect you have applied a filter in the sample pane or you may need to update the sample. Click on whole data and make sure the sample is updated.

Incidentally why are you using write_from_dataframe() instead of write_with_schema()? In Python recipes the recommended method is write_with_schema() so that the schema is updated as the data changes. You can still pass to it a data frame.

petertsui

Thank you for the feedback. I did as suggested and updated the sample, but still no data show up.

You suggestion to use the write_with_schema work when writing the dataframe to a filesystem_managed dataset.

The reason I used write_from_dataframe is because I also wanted to write the data into an Oracle database where I predefined the table structure. I needed to put the "response" column as a CLOB. I tried changing the "Table creation mode" and set it to Manually define without success.