When extracting date components from a date, year changes from double integer to double decimal
when extracting the date components from a date in a prepare recipe, it shows as being a double integer format but once i run the recipe, the ouput dataset has the year column as a double decimal format.
is there any way to keep it as a double integer?
i have tried a formula to format it and tried to round the date with no luck
thanks!
Operating system used: Windows 11
Best Answer
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,757 NeuronOptionsHave a read:
I am guessing you have some data not shown in your sample that is causing Dataiku to change the storage type fo your columns.
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,757 NeuronOptions
I can't reproduce your issue. Please post how you are extracting the components from the date. I have a Date field in a Prepare recipe, I click on its column heading and select "Extract dat components". Dataiku will then extract the fields as bigint (see screen shot below).
Also I think you are confusing the storage type with the meaning in Dataiku so I recommend you read this part of the documentation. In general you should be concerned with the storage type (top).
-
after double checking, the problem isnt from the 'extract date components', its from the next prepare recipe that the format changes. but in this recipe there is no work done on the year column
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,757 NeuronOptions
What is the storage type of the columns when they are extracted? Again I can't reproduce what you say it's happening. My columns are gigint and remain always bigint. Also your column names suggest you may be extracting the values manually. Please confirm exactly how you extract the date components.
-
extracting the date component, and the month and year column shows up as string integer/bigint integer respectively.
after this prepare recipe, there are other recipes used on this dataset, none of which do anything to to year column.
the dataset is then shared to another work zone and on the second prepare recipe, the format changes to double decimal.
-
i cant show the actual data due to confidentiality concerns.