Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hi, I am trying to write a pySpark.ml model (Pipeline model) from a pyspark recipe to a folder located in Server's filesystem storage and I am getting access denied error. How can I write to a folder from a pyspark recipe.
pipeline = Pipeline(stages=[<listofencoders>])
# Fit and transform the data using the pipeline
pipeline_model = pipeline.fit(data)
pipeline_model.save(dataiku.Folder("model").get_path() + "/model")
Operating system used: Linux
Hi, you can't write to Dataiku folders directly like that. You need to use the upload_stream() method:
https://knowledge.dataiku.com/latest/code/managed-folders/concept-managed-folders.html