When importing datasets with columns stored as int, DSS seems to want to store them as double.

Using wip_discrete_jobs as an example we have columns that should be integers but are ingested as numeric.

For example,

Table:  wip_discrete_jobs

Column: wip_entity_id

Type:    numeric(38,15)

If I query the table

SELECT wip_entity_id, organization_id  FROM khw_eng.wip_discrete_jobs WHERE wip_entity_id = 34922

The result is as expected with a scale of 15:

If I take this table as an input and do a data prepare step without adding any transformations, I end up with different column type.

Schema:           rpcadw

Table:    wip_discrete_jobs_prepared

Column: wip_entity_id

Type:                   float8

If I query the table I end up NULL result:

SELECT wip_entity_id, organization_id  FROM rpcadw.wip_discrete_jobs_prepared WHERE wip_entity_id  = 349225

To return the result I must round instead:

               SELECT wip_entity_id, organization_id  FROM rpcadw.wip_discrete_jobs_prepared WHERE round(wip_entity_id)  = 349225

We can see that the conversion to float by datiaku has not preserved the true value.

I also tried not normalizing float/doubles:

But we have the same issue:

               SELECT wip_entity_id, organization_id  FROM rpcadw.wip_discrete_jobs_prepared2 WHERE round(wip_entity_id) = 349225