Problem with Spark enigine

NunziaCoppola

Hello,

when we read hive tables with engine spark version 2.4.7.7.1.7.2038-1 on CDP version 7.1.7 we have a problem with type date, there's a shift forward of 10 min and 4 s as it is showns in the attached file.

Could you help us?

We have already tried to add:

spark.sql.legacy.parquet.int96RebaseModeInWrite --> CORRECTED

spark.sql.parquet.datetimeRebaseModeInRead -> CORRECTED

-----------------------------------------------------

spark.sql.legacy.parquet.int96RebaseModeInRead --> CORRECTED

spark.sql.legacy.parquet.int96RebaseModeInWrite --> CORRECTED

-----------------------------------------------------

spark.sql.legacy.parquet.datetimeRebaseModeInWrite --> CORRECTED

spark.sql.legacy.parquet.datetimeRebaseModeInRead --> CORRECTED

spark.sql.legacy.timeParserPolicy --> LEGACY

----------------------------------------------------

without any positive results!

Thanks

---

Operating system used: Red Hat Enterprise 8.6