OUTPUT_DATA_BAD_TYPE - resulting in loss of rows in data
Noah
Registered Posts: 43 ✭✭✭✭
so I have a dataset in an S3 connection that I am enriching with partition information and writing to a HDFS connection. I am losing 1100 records due to bad datatype.
How can I override this?
I tried by adding something to the prepare recipe where I use a formula ToNumber(Field1) saved as Field1_. I deleted Field1 then renamed Field1_ to Field1. That did not change anything
Answers
-
Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 317 Dataiker
Hi @nshapir2
,
We would need to review the job diagnostic and possibly a sample of the input data in order to advise on the best steps to resolve the issue, since it seems like there is a problem parsing your underlying data. Please feel free to open a support ticket with a job diagnostic and a sample of the data, and we can review it!
Thanks,
Sarina