Parse custom date no longer works after upgrade

dvancamp
dvancamp Registered Posts: 2 ✭✭✭✭

I upgraded from 3.x to 4.x DSS and one of my previously working cleanup scripts stopped parsing the date.

Looks like it checks out there but when processing we see this warning:


[12:49:25] [WARN] [dku.shaker.date] - SHAKER_BAD_DATE: Failed to parse '2017-02-13t07:27:41 cst' using format 'yyyy-MM-dd'T'HH:mm:ss z', at position 20

So, that format in the warning won't work because it has internal single quotes and is surrounded by single quotes. Not sure if that is something that would have changed between versions. I tried various escaping \'T\', "T", \T. I can't seem to get it to parse this date the way it did before... Any clues to what I'm doing wrong?

EDIT: I used a regex replace to pop the 'T' out of that column. Still not parsing date.

Answers

  • Alex_Reutter
    Alex_Reutter Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer Posts: 105 ✭✭✭✭✭✭✭

    Hi, with a little testing, I would say it's the lowercase cst that the parser objects to in that row.

  • dvancamp
    dvancamp Registered Posts: 2 ✭✭✭✭
    Thanks Alex.

    I checked that. All of my source file dates are formatted properly with uppercase TZ data. The green match bar goes all the way across for me. It appears to be lowercasing them while processing (or at least when displaying the warning). The entire list gets the error.

    SHAKER_BAD_DATE occured 129947 times
    Failed to parse '2017-02-13 07:27:41 cst' using format 'yyyy-MM-dd HH:mm:ss z', at position 20
    Failed to parse '2018-01-30 15:57:14 cst' using format 'yyyy-MM-dd HH:mm:ss z', at position 20
    Failed to parse '2017-02-20 20:06:05 cst' using format 'yyyy-MM-dd HH:mm:ss z', at position 20
    Failed to parse '2017-10-14 12:47:31 cdt' using format 'yyyy-MM-dd HH:mm:ss z', at position 20

    This is with the modified date where I removed the 'T' from the center.

    Since this was an existing project and data source from before the upgrade I'll do over with a new project to see if something didn't translate or if I have a problem with my installation...
  • Alex_Reutter
    Alex_Reutter Alpha Tester, Dataiker Alumni, Dataiku DSS Core Designer Posts: 105 ✭✭✭✭✭✭✭
    Hm... what 4.x version are you using? I was testing in 4.2, and it was not lowercasing while processing -- in the screenshot I posted, the input sample shows capital CST in the rows where it's capitalized, and lowercase in the rows where it's not.
Setup Info
    Tags
      Help me…