Partitioned Dataset in Jupyter Notebook

jaga
Level 1
Partitioned Dataset in Jupyter Notebook

Hi,

I am not able to write in the partitions of the dataset.

It is showing schema compatibility error.

Thank you for help


Operating system used: Windows

0 Kudos
1 Reply
JordanB
Dataiker

Hi @jaga,

I would need to see the actual error, which should mention where the incompatibility is. However, please note that this can happen because Pandas, by default, infers data types from data. 

For such cases where you may have mixed data in the same column, the fix is to use "infer_with_pandas=False" with the get_dataframe() method like below:
 
dataset = dataiku.Dataset("dataset_name")
df = dataset.get_dataframe(infer_with_pandas=False)
If this does not help, please provide the full error message/stack trace.
 
Thanks!
Jordan
0 Kudos