Data Values getting changed while exporting to SharePoint Online Folder or on Download

Options
Prakhyath
Prakhyath Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 10 ✭✭✭✭

Hi Team,

We have a scenario where the transformed dataset is exported to SharePoint online folders, email reports and flat downloads from end users. We found that certain values in a text field gets changed automatically as shown below,

Actual Value: 32005195642052503618

After Export: 32005195642052500000 (Last Four Characters are changed to 0)

This happens when the value has more that 14 numeric characters in text field.

The intermediate datasets are in Snowflake and we use In-database (SQL) engine for recipe's.

PFA Screenshot from Dataiku and Excel Download.


Operating system used: Windows 64 bit

Tagged:

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,708 Neuron
    Options

    This may be an issue with floating-point arithmetics, have a look at this post:

    https://community.dataiku.com/t5/Using-Dataiku/Rounding-issues/td-p/1985

    If the field is an account number then make sure it is defined as a string/text field from the time you input it into Dataiku to the time you output it.

  • Prakhyath
    Prakhyath Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 10 ✭✭✭✭
    Options

    The field is ingested as text field and it's not changed to other type in the transformations. But still I am facing the issue only when it's exported or downloaded.

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,708 Neuron
    Options

    I don't think you are ingesting the field as a string otherwise you wouldn't be missing characters. I can see on your Dataiku screen shot that someone locked the meaning of the column as Text. That means somewhere in your flow Dataiku must be coming up with the meaning as double which means the field is not a string. So to be sure that you ingest this field as a text make sure you apply a "to string" function in your data source so that Dataiku gets this field from your data source as a string. Then make sure it remains a string all the way through your flow.

Setup Info
    Tags
      Help me…