I've create a custom recipe with a Dataset and Managed Folder as input and a Dataset as output. The recipe runs well on DSS but it doesn't work when its configured to run on a Containerized Execution. These are the steps i took and also the error i received after running the receipt.
FileNotFoundError: [Errno 2] No such file or directory: '/mtn/...some path/57CFjKsj'
As far as i can tell, the Containerized Execution cannot see the files in Managed Folder. Could you advise what is going on?
To run a recipe on Containerized Execution when using managed folders as input/output, you will need to use the Dataiku API for reading/writing files.
In other words, the "regular" local filesystem API which you were using with local execution cannot work anymore. Hence, you will need to use the following methods to interact with the managed folder:
The same APIs also exist in R, as documented here:
Hope it helps,