Issue converting dataset to JSON

xXdbrzXx
Level 1
Issue converting dataset to JSON

Hi All,

I'm trying to convert a dataset into a JSON format by using a python script and then calling it via Postman to use it externally. I am running into an issue when calling to Postman, it never seems to read the formatting correctly and it prints a series of arrays (attached image)Screenshot 2024-05-01 143217.jpg

 

Here's the code that I'm using:

import dataiku
import pandas as pd
from dataiku import pandasutils as pdu

Predict = dataiku.Dataset("Predict")
Predict_df = Predict.get_dataframe()
json_data = Predict_df.to_dict(orient='records')

import json
clean_json_str = json.dumps(json_data, indent=None)  
JSON_predict_df = pd.DataFrame({'json_data': [clean_json_str]})

JSON_predict = dataiku.Dataset("JSON_Predict")
JSON_predict.write_with_schema(JSON_predict_df)

Can anyone provide some insight as to what I'm doing wrong?

Best,

Dylan

0 Kudos
1 Reply
Turribeach

You can't write a dataset back to JSON format. The format that Dataiku uses to stores datasets is not something you can control. If you need to send a dataset as a JSON format to an external API you can convert it on the fly and send it as JSON there and then, do not write it back to the Dataiku dataset. So in your code once you have clean_json_str you should call your external API right there. 

0 Kudos