OUTPUT_DATA_BAD_TYPE - resulting in loss of rows in data

Noah
Noah Registered Posts: 43 ✭✭✭✭

so I have a dataset in an S3 connection that I am enriching with partition information and writing to a HDFS connection. I am losing 1100 records due to bad datatype.

How can I override this?

I tried by adding something to the prepare recipe where I use a formula ToNumber(Field1) saved as Field1_. I deleted Field1 then renamed Field1_ to Field1. That did not change anything

Answers

  • Sarina
    Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 317 Dataiker

    Hi @nshapir2
    ,

    We would need to review the job diagnostic and possibly a sample of the input data in order to advise on the best steps to resolve the issue, since it seems like there is a problem parsing your underlying data. Please feel free to open a support ticket with a job diagnostic and a sample of the data, and we can review it!

    Thanks,
    Sarina

Setup Info
    Tags
      Help me…