Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
May I know how I can use Pyspark recipe to save my pyspark dataframe as csv file to a output managed folder?
I have searched in the community, but most of posts cover pandas dataframe only....
A Pyspark DataFrame is by definition a dataframe that only exists on your Pyspark engine so in order to save it in Dataiku you first need to bring to memory. You can do that by calling the toPandas() method.
A Pyspark DataFrame is by definition a dataframe that only exists on your Pyspark engine so in order to save it in Dataiku you first need to bring to memory. You can do that by calling the toPandas() method.