Syncing S3 data to Snowflake and skipping malformed rows

yashpuranik Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2022, Neuron 2023 Posts: 69 Neuron

Hi All,

Suppose I have a malformed CSV in S3.


If I try to sync to Snowflake using S3 to Snowflake engine, I get the following error.


Of course I can get around it if I select DSS as the engine, but I don't want to unnecessarily use up a lot of disk space (even if temporarily) because I am looking to move massive datasets.

How do I modify the behaviour of the S3 to Snowflake stream as suggested in the error logs?


  • Sarina
    Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer Posts: 315 Dataiker

    Hi @yashpuranik

    We have an existing request in our backlog to allow for use of additional Snowflake ON_ERROR options. At the moment, your only option would be to clean the data before the sync.

    I hope that information is helpful.


Setup Info
      Help me…