Sync array column to PostgreSQL table

valentinaprotti
valentinaprotti Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Registered Posts: 2 ✭✭✭

Hi everyone,

I am trying to sync the dataset resulting from the last step of my flow to a PostgreSQL table. However, the sync recipe fails with the following:

Invalid argument

An invalid argument has been encountered : in act.compute_output_email_NP: Can't handle column type 'ARRAY' in 'PostgreSQL': see column 'attach' from dataset 'NLP_EMAIL_PARSER.output_email'.

I tried to explicitly set the column's storage type to string, but it doesn't seem to work.

Any ideas on how to solve this? If necessary, adding another recipe in the middle would be ok.

Thanks!


Operating system used: MacOS

Best Answer

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,212 Dataiker
    Answer ✓

    Hi @valentinaprotti
    ,

    From your error, it looks like you may have the option "maintain strict schema" in your Sync recipe. For that, to work you would need to change the schema of the input dataset, which would then propagate automatically to the output dataset.

    If you change sync recipe settings to "Free output schema" it should work with your current configuration after you edited the output dataset schema from array to string/Text/

    Screenshot 2021-12-18 at 12.15.48.png

    Screenshot 2021-12-18 at 12.11.42.png

    Let me know if that helps.

Setup Info
    Tags
      Help me…