Submit your innovative use case or inspiring success story to the 2023 Dataiku Frontrunner Awards! LET'S GO

Filtering Data based on a created EST time Column from a UTC Column

leeIdoine
Level 2
Filtering Data based on a created EST time Column from a UTC Column

Hello, 

I have a dataset where the data has a time stamp that comes in as UTC that I convert to EST. I did this by format date recipe. 

 

 

pic1.JPG

I am trying to filter the data now, by filtering the data from 8AM-9PM, for 5/24. However, the filter is returning values that are outside of this specified time frame. (example: its returning values that have an EST time stamp of 5AM). Additionally, when I hover over the EST column and click analyze, it says that the max value in the data set is "2023-05-24:20:59:50". However, when I scroll to the bottom of the dataset, the maximum value it shows 2023-05-24:16:59:50. And when I export the data set as a CSV, the maximum value in the EST column is 16:59:50. So well before 9PM 

Would anyone have any idea of what is going on here? I am confused as to why the filter is returning values that come before 8AM, and I am also confused that its saying that the dataset has values up until 9PM. But I can not see them in the dataset, and I can not get it to export in a CSV. 

(I am not sampling the dataset in the filter) I have no sampling selected. 

 

Thanks, 

 

 

0 Kudos
0 Replies