Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I am using 3 excel files. Uisng prepare recipe I have formatted all 3 excels to make it dataIKU compatible. And stacked them together. I am fine until this step, after this step I have created output recipe to send dataset to Tableau. And I got this error
Hi @Bhargavi27,
This indeed appears to be due to the underlying Tableau tableauhyperapi package handles dates:
https://community.tableau.com/s/question/0D54T00000F33cOSAR/inserting-null-and-textbased-datetimes-w...
To handle this, you can either convert the column from the type timestamp to string. Alternatively, you can run a Python recipe on your dataset to set a dummy value for any empty timestamp columns and leave the type of the column as a timestamp type. For example:
# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu
# Read recipe inputs
input_dataset = dataiku.Dataset("<INPUT_DATASET>")
df = input_dataset.get_dataframe()
# Replace 'nan' with NaN (if 'nan' is a string)
df = df.replace('nan', np.nan)
# Replace NaN and empty/null values with '1970-01-01T00:00:00.000Z'
date_to_replace = '1970-01-01T00:00:00.000Z'
output_df = df.fillna(date_to_replace)
output_dataset = dataiku.Dataset("<OUTPUT_DATASET>")
output_dataset.write_with_schema(output_df)
If you have a way of easily identifying the columns you want to set the na values with, you can add a section to only apply this to certain columns:
columns ['value1', 'value2'.....]
for column in columns:
output_df[column] = df[column].fillna(date_to_replace)
I hope that helps!
Thanks,
Sarina