Parse custom date no longer works after upgrade

dvancamp
Level 1
Parse custom date no longer works after upgrade

I upgraded from 3.x to 4.x DSS and one of my previously working cleanup scripts stopped parsing the date.





 



Looks like it checks out there but when processing we see this warning:




[12:49:25] [WARN] [dku.shaker.date] - SHAKER_BAD_DATE: Failed to parse '2017-02-13t07:27:41 cst' using format 'yyyy-MM-dd'T'HH:mm:ss z', at position 20


 



So, that format in the warning won't work because it has internal single quotes and is surrounded by single quotes. Not sure if that is something that would have changed between versions. I tried various escaping \'T\', "T", \T. I can't seem to get it to parse this date the way it did before... Any clues to what I'm doing wrong?



 



EDIT: I used a regex replace to pop the 'T' out of that column. Still not parsing date.

0 Kudos
3 Replies
Alex_Reutter
Dataiker Alumni

Hi, with a little testing, I would say it's the lowercase cst that the parser objects to in that row.



0 Kudos
dvancamp
Level 1
Author
Thanks Alex.

I checked that. All of my source file dates are formatted properly with uppercase TZ data. The green match bar goes all the way across for me. It appears to be lowercasing them while processing (or at least when displaying the warning). The entire list gets the error.

SHAKER_BAD_DATE occured 129947 times
Failed to parse '2017-02-13 07:27:41 cst' using format 'yyyy-MM-dd HH:mm:ss z', at position 20
Failed to parse '2018-01-30 15:57:14 cst' using format 'yyyy-MM-dd HH:mm:ss z', at position 20
Failed to parse '2017-02-20 20:06:05 cst' using format 'yyyy-MM-dd HH:mm:ss z', at position 20
Failed to parse '2017-10-14 12:47:31 cdt' using format 'yyyy-MM-dd HH:mm:ss z', at position 20

This is with the modified date where I removed the 'T' from the center.

Since this was an existing project and data source from before the upgrade I'll do over with a new project to see if something didn't translate or if I have a problem with my installation...
0 Kudos
Alex_Reutter
Dataiker Alumni
Hm... what 4.x version are you using? I was testing in 4.2, and it was not lowercasing while processing -- in the screenshot I posted, the input sample shows capital CST in the rows where it's capitalized, and lowercase in the rows where it's not.
0 Kudos