Partitioned Dataset in Jupyter Notebook

jaga
jaga Registered Posts: 2

Hi,

I am not able to write in the partitions of the dataset.

It is showing schema compatibility error.

Thank you for help


Operating system used: Windows

Tagged:

Answers

  • JordanB
    JordanB Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 296 Dataiker

    Hi @jaga
    ,

    I would need to see the actual error, which should mention where the incompatibility is. However, please note that this can happen because Pandas, by default, infers data types from data.

    For such cases where you may have mixed data in the same column, the fix is to use "infer_with_pandas=False" with the get_dataframe() method like below:
    dataset = dataiku.Dataset("dataset_name")
    df = dataset.get_dataframe(infer_with_pandas=False)
    If this does not help, please provide the full error message/stack trace.
    Thanks!
    Jordan
Setup Info
    Tags
      Help me…