Want to Stop Rebuilding "Expensive" Parts of your Flow? Explicit Builds are the Answer!READ MORE

Sync array column to PostgreSQL table

Solved!
valentinaprotti
Level 1
Sync array column to PostgreSQL table

Hi everyone,

I am trying to sync the dataset resulting from the last step of my flow to a PostgreSQL table. However, the sync recipe fails with the following:

 Invalid argument

An invalid argument has been encountered : in act.compute_output_email_NP: Can't handle column type 'ARRAY' in 'PostgreSQL': see column 'attach' from dataset 'NLP_EMAIL_PARSER.output_email'.

I tried to explicitly set the column's storage type to string, but it doesn't seem to work.

Any ideas on how to solve this? If necessary, adding another recipe in the middle would be ok.

Thanks!


Operating system used: MacOS

0 Kudos
1 Solution
AlexT
Dataiker
Dataiker

Hi @valentinaprotti,

From your error, it looks like you may have the option "maintain strict schema" in your Sync recipe. For that, to work you would need to change the schema of the input dataset, which would then propagate automatically to the output dataset.

If you change sync recipe settings to "Free output schema" it should work with your current configuration after you edited the output dataset schema from array to string/Text/

Screenshot 2021-12-18 at 12.15.48.png

 

Screenshot 2021-12-18 at 12.11.42.png

Let me know if that helps. 

View solution in original post

1 Reply
AlexT
Dataiker
Dataiker

Hi @valentinaprotti,

From your error, it looks like you may have the option "maintain strict schema" in your Sync recipe. For that, to work you would need to change the schema of the input dataset, which would then propagate automatically to the output dataset.

If you change sync recipe settings to "Free output schema" it should work with your current configuration after you edited the output dataset schema from array to string/Text/

Screenshot 2021-12-18 at 12.15.48.png

 

Screenshot 2021-12-18 at 12.11.42.png

Let me know if that helps. 

Labels

?
Labels (2)
A banner prompting to get Dataiku