Discover all of the brand-new features and improvements to existing capabilities in the Dataiku 11.3 updateLET'S GO

Dataframes not reading NA country code in python receipe

Solved!
Ankur30
Level 2
Dataframes not reading NA country code in python receipe

Hi , 

Good Evening!!

I am reading a pandas dataframe from dataiku dataset and filtering on county code == 'NA' but my dataframe is not returning any value.

 

Is there any workaround for this. Kindly help!

 

Regards,

Ankur.

 

 

 

0 Kudos
1 Solution
AlexT
Dataiker

This behavior is expected with Pandas.

Starting with DSS 9.0.4 or later we added support for pandas parameters keep_default_na.

Which would allow you to add keep_default=False to your get_dataframe() function to handle this situation.

One possible workaround for the previous version would be:

1) Export to CSV to a managed folder using the Visual Export recipe and read the CSV with pandas from the managed folder:

Use sample code this works only with local filesystem managed folder : 

 

import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu

# Read recipe inputs
managed_folder = dataiku.Folder("D0gTVBY3")
path = managed_folder.get_path()
filename = "country_prepared.csv"
filepath = path + "/" + filename

df = pd.read_csv(filepath, keep_default_na=False) 

 

 

View solution in original post

0 Kudos
2 Replies
AlexT
Dataiker

This behavior is expected with Pandas.

Starting with DSS 9.0.4 or later we added support for pandas parameters keep_default_na.

Which would allow you to add keep_default=False to your get_dataframe() function to handle this situation.

One possible workaround for the previous version would be:

1) Export to CSV to a managed folder using the Visual Export recipe and read the CSV with pandas from the managed folder:

Use sample code this works only with local filesystem managed folder : 

 

import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu

# Read recipe inputs
managed_folder = dataiku.Folder("D0gTVBY3")
path = managed_folder.get_path()
filename = "country_prepared.csv"
filepath = path + "/" + filename

df = pd.read_csv(filepath, keep_default_na=False) 

 

 

0 Kudos
Ankur30
Level 2
Author

Thanks alot Alex 🙂

0 Kudos

Labels

?
Labels (4)
A banner prompting to get Dataiku