Loading a Spark Dataframe to snowflake , python3.8 upgrade
verleger
Partner, Registered Posts: 2 Partner
How do I write a spark Dataframe to Snowflake table from Jupyter notebook in Dataiku ?
so much document, very little examples !
Thanks
Answers
-
Hi,
Assuming that "myoutput" is the name of the Snowflake dataset in which you want to write, and that "df" is the Spark dataframe, you would write:
import dataiku from dataiku import spark as dkuspark myoutput = dataiku.Dataset("myoutput") dkuspark.write_with_schema(out, df)
https://doc.dataiku.com/dss/latest/code_recipes/pyspark.html
Best,