Want to store my output column with decimal data type as storage datatype

SUSHIL

Hi,

I have an requirment where my column has an value with decimal format with precision.

When prepare recipe it reads as double and value is displaying properly with proper decimal values.

When we run the recipe in the output the value is getting changed with the scientific notation.

Why the output format is getting changed automatically with scientifc notation and in final output in hive table also value gets stored with scientific notation.

How to avoid these type of issue. Can anyone help and guide us?

Miguel Angel

Hi,

In a prepare recipe DSS automatically fills the schema of the output dataset at creation time.
This schema is inferred by looking at the data in the sample. DSS automatically tries to use the “best” storage type that matches the data..

Scientific notation is a valid format for doubles, as seen here: https://knowledge.dataiku.com/latest/kb/data-prep/prepare-recipe/decimal-notations.html#

If you want to control the format of the numbers at the output of the prepare recipe, the solution is to turn them into strings. See the following article with an in-depth explanation about it: https://knowledge.dataiku.com/latest/kb/data-prep/prepare-recipe/How-to-remove-scientific-notation-in-a-column.html