Cannot use a managed folder as an input of a python recipe created from python code using builder
Hi team,
I currently encountered an urgent issue with our project automation pipeline. In a Python code recipe, I used builder to create a new python recipe, and have added a dataset as input, 3 new datasets as output. In order to save some model objects to a managed folder, I need to add an existing folder "Pickle_Files" as input of the newly created python recipe. (I can't add the folder through the UI, because all the steps and files are automatically generated by the previous python script).
I didn't have issues when I used files as inputs. So is there a way to add a managed folder as input of a python recipe in code?
Thanks,
Anna
Operating system used: Mac
Answers
-
When I created a new managed folder in the python code, moved it to the current working zone, and then tried to use this new folder as input of another new python recipe, it showed a new error:
code:
new_folder = project.create_managed_folder("test")
new_folder.move_to_zone("vdVhYpH")
builder.with_input(new_folder)
error:
TypeError: Object of type DSSManagedFolder is not JSON serializable
-
I found the solution....
-
CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭
Hey, @AnnaZ
that's great! Would it be possible to share what your solution is? That way if someone else has a similar issue in the future they would be able to utilize your solution for their own benefit. Thanks! -
Same problem! Wish the solution was shared.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,167 Neuron
Please start a new thread, post your code and the error you get and we will help you.
-
No need, I just had to make sure the dataset name in the input/output tab matched what was in my python recipe.
-
@pnw_ds
Can you please share the code snippet as I've a similar issue -
Hi All,
I'm trying to export my final df of a python recipe to a managed folder & I get the below mentioned error:
Job failed: Error in Python process: At line 35: <class 'Exception'>: Managed folder EyABfYFjV cannot be used : declare it as input or output of your recipe
Any code snippet via which I can push my df data into a csv file & then when I go back to the recipes it should show the managed folder icon wherein my file should be present.