Survey banner
Switching to Dataiku - a new area to help users who are transitioning from other tools and diving into Dataiku! CHECK IT OUT

Faster option for writing small data to Snowflake from code studio

Jonny2650
Level 1
Faster option for writing small data to Snowflake from code studio

I have a little python streamlit webapp I've made in code studio that outputs data generated to a snowflake table that's in the same project. The way I'm attempting to do it (code below) is taking up to 2 minutes to upload the data but it's only ever a couple of rows of data (<10), 20 columns and no large fields. I would have thought for data this small that it would take under 30s. Is there a faster way of doing this? 

with st.spinner('Uploading to Snowflake...'):
    output_ds = dataiku.Dataset("output_dataset")
    output_ds.write_with_schema(df_to_write)
st.success('Successful Upload into Snowflake!', icon="")
0 Kudos
1 Reply
Turribeach

How long does it take to run in a normal Python recipe?

0 Kudos

Labels

?
Labels (2)
A banner prompting to get Dataiku