Failed to export row error

Bhargavi27 Dataiku DSS Core Designer, Registered Posts: 7

I am using 3 excel files. Uisng prepare recipe I have formatted all 3 excels to make it dataIKU compatible. And stacked them together. I am fine until this step, after this step I have created output recipe to send dataset to Tableau. And I got this error

Job failed: Failed to export rows : <class 'ValueError'> : Got an invalid value 'nan' for column "Original_Audit_Date", it must be a Timestamp or a datetime instance


  • Sarina
    Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer Posts: 315 Dataiker
    edited July 17

    Hi @Bhargavi27

    This indeed appears to be due to the underlying Tableau
    tableauhyperapi package handles dates:

    To handle this, you can either convert the column from the type timestamp to string. Alternatively, you can run a Python recipe on your dataset to set a dummy value for any empty timestamp columns and leave the type of the column as a timestamp type. For example:

    # -*- coding: utf-8 -*-
    import dataiku
    import pandas as pd, numpy as np
    from dataiku import pandasutils as pdu
    # Read recipe inputs
    input_dataset = dataiku.Dataset("<INPUT_DATASET>")
    df = input_dataset.get_dataframe()
    # Replace 'nan' with NaN (if 'nan' is a string)
    df = df.replace('nan', np.nan)
    # Replace NaN and empty/null values with '1970-01-01T00:00:00.000Z'
    date_to_replace = '1970-01-01T00:00:00.000Z'
    output_df = df.fillna(date_to_replace)
    output_dataset = dataiku.Dataset("<OUTPUT_DATASET>")

    If you have a way of easily identifying the columns you want to set the na values with, you can add a section to only apply this to certain columns:

    columns ['value1', 'value2'.....]
    for column in columns:
       output_df[column] = df[column].fillna(date_to_replace)

    I hope that helps!


Setup Info
      Help me…