Writting a column with date format in Big Query
Options
Vale_NCH
Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 6 ✭
Hi everyone,
I'm using Dataiku 11.2 with Big Query and GCS.
I need to upload to Big Query (using a sync recipe) a table with a column in DATE format, the problem is that Dataiku uses TIMESTAMP instead of DATE format, so when i do the sync, the format of the resulting table in Big Query is TIMESTAMP.
If you need further information, I'll be pleased to give you more details about this.
Best regards,
V.
Answers
-
JordanB Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 293 DataikerOptionsHi @Vale_NCH
,In DSS 11.1.0, we changed the way BigQuery creates BigQuery tables for managed datasets to match what the other recipes do. DSS will now only create BigQuery tables with TIMESTAMP columns and no longer with DATE or DATETIME columns. This prevents issues in downstreams recipes where DSS is not capable of correctly reading back the DATE or DATETIME columns (i.e. without unexpected timeshifts).DSS 11.1.0 release notes: https://doc.dataiku.com/dss/latest/release_notes/11.html#version-11-1-0-october-21st-2022It's this line in the release notes* Fixed issue with date types coming from BigQueryPlease let me know if you have any questions.Thanks!Jordan