Do you know the History of Data Science? READ MORE

How Dataiku reads date columns from a text file.

Solved!
PANKAJ
Level 3
Level 3
How Dataiku reads date columns from a text file.

When I uploaded two text files in Dataiku in which date columns are there in the format MMddyyyy along with few empty columns. Surprisingly Dataiku reads the date column of the 1st table as Date(Unparsed) type but text type for the second. Now I can easily parse the date column in the 1st table but what about the 2nd one? When I change the type of that from text to date it makes all rows of that column red. So how can I change that into a parsed date format column?

Help, please.

0 Kudos
1 Solution
Ignacio_Toledo

Hi @PANKAJ,

Let's get into some details (if I understand your use case correctly).

First, you need to understand the difference between "storage type" and "meaning". When in the first table the date column is shown as "Date(Unparsed)" you are talking about the "meaning" that Dataiku is giving to that column. Now, because is (Unparsed) you are still required to do an explicit parsing with a "Prepare Recipe" to actually get Dataiku to understand the column as a "Date", else you won't be able to use the date functions to interact with that column.

So, even if the second table, for some unknown reason that might be related the contents in one particular cell perhaps, is detected with meaning "Text", you will still need to do the parsing using a "Prepare Recipe" (see this link: https://doc.dataiku.com/dss/latest/preparation/processors/date-parser.html#)

After you do the parse date step in the prepare recipe, both or your columns will be now given the meaning "Date" by DSS.

Hope this helps.

 

View solution in original post

3 Replies
CoreyS
Community Manager
Community Manager

Hi, @PANKAJ! Can you provide any further details on the thread to assist users in helping you find a solution (insert examples like DSS version etc.) Also, can you let us know if you’ve tried any fixes already?This should lead to a quicker response from the community.

Looking for more resources to help you use DSS effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as ‘Accepted Solution’ to help others like you!
0 Kudos
Ignacio_Toledo

Hi @PANKAJ,

Let's get into some details (if I understand your use case correctly).

First, you need to understand the difference between "storage type" and "meaning". When in the first table the date column is shown as "Date(Unparsed)" you are talking about the "meaning" that Dataiku is giving to that column. Now, because is (Unparsed) you are still required to do an explicit parsing with a "Prepare Recipe" to actually get Dataiku to understand the column as a "Date", else you won't be able to use the date functions to interact with that column.

So, even if the second table, for some unknown reason that might be related the contents in one particular cell perhaps, is detected with meaning "Text", you will still need to do the parsing using a "Prepare Recipe" (see this link: https://doc.dataiku.com/dss/latest/preparation/processors/date-parser.html#)

After you do the parse date step in the prepare recipe, both or your columns will be now given the meaning "Date" by DSS.

Hope this helps.

 

View solution in original post

PANKAJ
Level 3
Level 3
Author

Ohh...got it. Thank You so much.

A banner prompting to get Dataiku DSS