Failed to export row error

Level 2
Failed to export row error

I am using 3 excel files. Uisng prepare recipe I have formatted all 3 excels to make it dataIKU compatible. And stacked them together.  I am fine until this step, after this step I have created output recipe to send dataset to Tableau. And I got this error



Job failed: Failed to export rows : <class 'ValueError'> : Got an invalid value 'nan' for column "Original_Audit_Date", it must be a Timestamp or a datetime instance 

0 Kudos
1 Reply

Hi @Bhargavi27,

This indeed appears to be due to the underlying Tableau
 tableauhyperapi package handles dates:

To handle this, you can either convert the column from the type timestamp to string. Alternatively, you can run a Python recipe on your dataset to set a dummy value for any empty timestamp columns and leave the type of the column as a timestamp type. For example:

# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu

# Read recipe inputs
input_dataset = dataiku.Dataset("<INPUT_DATASET>")
df = input_dataset.get_dataframe()

# Replace 'nan' with NaN (if 'nan' is a string)
df = df.replace('nan', np.nan)

# Replace NaN and empty/null values with '1970-01-01T00:00:00.000Z'
date_to_replace = '1970-01-01T00:00:00.000Z'
output_df = df.fillna(date_to_replace)

output_dataset = dataiku.Dataset("<OUTPUT_DATASET>")

If you have a way of easily identifying the columns you want to set the na values with, you can add a section to only apply this to certain columns:

columns ['value1', 'value2'.....]
for column in columns:
   output_df[column] = df[column].fillna(date_to_replace)

I hope that helps! 


0 Kudos