Failed to export row error

Bhargavi27

I am using 3 excel files. Uisng prepare recipe I have formatted all 3 excels to make it dataIKU compatible. And stacked them together. I am fine until this step, after this step I have created output recipe to send dataset to Tableau. And I got this error

Job failed: Failed to export rows : <class 'ValueError'> : Got an invalid value 'nan' for column "Original_Audit_Date", it must be a Timestamp or a datetime instance

Sarina

Hi @Bhargavi27
,

This indeed appears to be due to the underlying Tableau tableauhyperapi package handles dates:
https://community.tableau.com/s/question/0D54T00000F33cOSAR/inserting-null-and-textbased-datetimes-with-inserter-method-in-python-hyper-api

To handle this, you can either convert the column from the type timestamp to string. Alternatively, you can run a Python recipe on your dataset to set a dummy value for any empty timestamp columns and leave the type of the column as a timestamp type. For example:

# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu

# Read recipe inputs
input_dataset = dataiku.Dataset("<INPUT_DATASET>")
df = input_dataset.get_dataframe()

# Replace 'nan' with NaN (if 'nan' is a string)
df = df.replace('nan', np.nan)

# Replace NaN and empty/null values with '1970-01-01T00:00:00.000Z'
date_to_replace = '1970-01-01T00:00:00.000Z'
output_df = df.fillna(date_to_replace)

output_dataset = dataiku.Dataset("<OUTPUT_DATASET>")
output_dataset.write_with_schema(output_df)

If you have a way of easily identifying the columns you want to set the na values with, you can add a section to only apply this to certain columns:

columns ['value1', 'value2'.....]
for column in columns:
   output_df[column] = df[column].fillna(date_to_replace)

I hope that helps!

Thanks,
Sarina