The Dataiku Frontrunner Awards have just launched to recognize your achievements! Submit Your Entry

When reading dataiku dataset, can we read empty string as empty string instead of float NaN?

Blossom
Level 1
When reading dataiku dataset, can we read empty string as empty string instead of float NaN?

Hello  community,

I'm developing dataiku recipes. I used a recipe to transform json response to dataframe and write the dataframe as output to dataiku. Pandas will fill not-exsiting-field for some rows with Float NaN automatically, which is not a problem. 

But when we read the csv file in dataiku, pandas will treat empty string as Float NaN as well, which has an impact to our work flow. I know that in pandas, there is parameter with which we can treat empty string as empty string. But it seems like this parameter doesn't exist in dataiku?

pd.read_csv('test.csv', keep_default_na=False)

 Solutions that I have tried:

1. Verify the schema and make sure that the schema is correct.

2. infer_with_pandas=False

3. Save the dataset in other format like parquet

But none of them works.

Do you have some suggestions on what I should do?

 

Regards

0 Kudos
0 Replies
A banner prompting to get Dataiku DSS