This is regarding the slowness of Dataiku write back to Redshift.
We followed this documentations:https://doc.dataiku.com/dss/latest/connecting/sql/redshift.html
Direct redshift load is really slow, we tried the approach suggested in the docs, first we loaded the data to S3 then we tried to load to Redshift internal table.
Loading to S3 was fast but we were not able to write data from S3 to Redshift.
We have a few questions regarding Dataiku Redshift integration:
Does anyone know what meta store uses Dataiku to write to S3? Is it the one that is set up for DSS?
Can we use separate IAM roles to write into S3 and the write back to redshift from S3?
How can we check the version of the jdbc driver that Dataiku uses for Redshift? Can we use a redshift driver that is capable of bulk insert directly to redshift?