You now have until September 15th to submit your use case or success story to the 2022 Dataiku Frontrunner Awards!ENTER YOUR SUBMISSION

Dataframes not reading NA country code in python receipe

Solved!
Ankur30
Level 2
Level 2
Dataframes not reading NA country code in python receipe

Hi , 

Good Evening!!

I am reading a pandas dataframe from dataiku dataset and filtering on county code == 'NA' but my dataframe is not returning any value.

 

Is there any workaround for this. Kindly help!

 

Regards,

Ankur.

 

 

 

0 Kudos
1 Solution
AlexT
Dataiker
Dataiker

This behavior is expected with Pandas.

Starting with DSS 9.0.4 or later we added support for pandas parameters keep_default_na.

Which would allow you to add keep_default=False to your get_dataframe() function to handle this situation.

One possible workaround for the previous version would be:

1) Export to CSV to a managed folder using the Visual Export recipe and read the CSV with pandas from the managed folder:

Use sample code this works only with local filesystem managed folder : 

 

import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu

# Read recipe inputs
managed_folder = dataiku.Folder("D0gTVBY3")
path = managed_folder.get_path()
filename = "country_prepared.csv"
filepath = path + "/" + filename

df = pd.read_csv(filepath, keep_default_na=False) 

 

 

View solution in original post

0 Kudos
2 Replies
AlexT
Dataiker
Dataiker

This behavior is expected with Pandas.

Starting with DSS 9.0.4 or later we added support for pandas parameters keep_default_na.

Which would allow you to add keep_default=False to your get_dataframe() function to handle this situation.

One possible workaround for the previous version would be:

1) Export to CSV to a managed folder using the Visual Export recipe and read the CSV with pandas from the managed folder:

Use sample code this works only with local filesystem managed folder : 

 

import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu

# Read recipe inputs
managed_folder = dataiku.Folder("D0gTVBY3")
path = managed_folder.get_path()
filename = "country_prepared.csv"
filepath = path + "/" + filename

df = pd.read_csv(filepath, keep_default_na=False) 

 

 

0 Kudos
Ankur30
Level 2
Level 2
Author

Thanks alot Alex 🙂

0 Kudos

Labels

?
Labels (4)
A banner prompting to get Dataiku