DSS auto save Json String To Array
tmgiang
Registered Posts: 15 ✭✭✭✭
I use Python to read data from Mongodb. But when I save to DSS, the json data automatically store as "array" like that:
[{u'keyed_data': [{u'indexed_EDLetterCode_orig': ...
(actually, the json data is array)
Before use DSS, I code the recipe on Jupyter Notebook and it works well (I use json_normalize to read json data to panda frame). But after DSS automatically store the json in "array" type, I can't use json normalize
I found that the preparation recipe can fold / unfold the json data to rows, but unsuccesfully.
How can I read the json data to rows? Can I force DSS does not store json data like "array" ?
[{u'keyed_data': [{u'indexed_EDLetterCode_orig': ...
(actually, the json data is array)
Before use DSS, I code the recipe on Jupyter Notebook and it works well (I use json_normalize to read json data to panda frame). But after DSS automatically store the json in "array" type, I can't use json normalize
I found that the preparation recipe can fold / unfold the json data to rows, but unsuccesfully.
How can I read the json data to rows? Can I force DSS does not store json data like "array" ?
Answers
-
Hi,
We have a native MongoDB connector: https://www.dataiku.com/learn/guide/connecting/dss-and-mongodb.html. Could you try to switch to it? Let us know if it solves the issue you are having with the custom Python connection.
Cheers,
Alex -
thanks your answer but I can not create data set with mongo connection (i created mongo connection already). Maybe I need to upgrade my account to premium
-
Hello,
Indeed, native MongoDB connection is not available in the Community edition. You can read more on this page: https://www.dataiku.com/dss/editions/.
To pursue with the custom Python connection, have you tried writing the dataframe resulting from pandas.io.json_normalize into a dataiku.Dataset using`write_from_dataframe`? Feel free to paste some code if that is OK with you.
