Survey banner
Share your feedback on the Dataiku documentation with this 5 min survey. Thanks! TAKE THE SURVEY

Faster option for writing small data to Snowflake from code studio

Jonny2650
Level 1
Faster option for writing small data to Snowflake from code studio

I have a little python streamlit webapp I've made in code studio that outputs data generated to a snowflake table that's in the same project. The way I'm attempting to do it (code below) is taking up to 2 minutes to upload the data but it's only ever a couple of rows of data (<10), 20 columns and no large fields. I would have thought for data this small that it would take under 30s. Is there a faster way of doing this? 

with st.spinner('Uploading to Snowflake...'):
    output_ds = dataiku.Dataset("output_dataset")
    output_ds.write_with_schema(df_to_write)
st.success('Successful Upload into Snowflake!', icon="")
0 Kudos
1 Reply
Turribeach

How long does it take to run in a normal Python recipe?

0 Kudos

Labels

?
Labels (2)
A banner prompting to get Dataiku