Plugin development to manage different files in a managed folder based on user-defined mapping.

[arrawat]

We are trying to develop a plugin which should be able to read `n` number of files from a managed folder and put `m` number of files back to same or different managed folder, here `n` is not equals to `m`.
To achieve that we have created a project and exported that as a plugin. But the issue is our flow is starting with managed folder which eventually is getting replaced in the project where it is being used. Snapshots attached below.

1st Snapshot: Flow maintained in Parent project
How flow is designed in original project.

2nd Snapshot: Usage of plugin in Usage project
How plugin is used in Usage project.

3rd Snapshot: Flow executed in Usage project (enabled Debug application in Plugin)
What flow is being used in debug instance of application. Note: original folder is being replaced by shared folder of Usage project.

Error that we are getting:

Error in Python process: At line 11: <class 'Exception'>: Managed folder D1****H* cannot be used : declare it as input or output of your recipe

Plugin Configuration:

{
   "projectExportManifest": {
      "exportUploads": false,
      "exportAllInputDatasets": false,
      "exportAllDatasets": false,
      "exportManagedFS": false,
      "exportAnalysisModels": false,
      "exportSavedModels": false,
      "exportModelEvaluationStores": false,
      "exportProjectResources": false,
      "exportLabelingTasks": false,
      "exportManagedFolders": false,
      "exportAllInputManagedFolders": false,
      "exportInsightsData": false,
      "includedDatasetsData": [],
      "includedSavedModels": [],
      "includedManagedFolders": [],
      "includedModelEvaluationStores": [],
      "includedCodeStudios": [],
      "includedLabelingTasks": [],
      "exportGitRepository": false
   },
   "instanceFeatures": {
      "showFlowNavLink": false,
      "showLabNavLink": false,
      "showCodeNavLink": false,
      "showSwitchToProjectViewButton": true,
      "showVersionControlFeatures": false
   },
   "useAppHomepage": true,
   "homepageSections": [],
   "id": "PROJECT_POC*********",
   "label": "POC - *******",
   "shortDesc": "",
   "tags": [],
   "useAsRecipeSettings": {
      "icon": "icon-puzzle-piece",
      "category": "Custom",
      "inputRoles": [
         {
            "objectId": "D1****H*",
            "roleLabel": "Input",
            "type": "MANAGED_FOLDER",
            "$touched.it.roleLabel": true,
            "$invalid": false
         }
      ],
      "outputRoles": [
         {
            "objectId": "X****J*t",
            "roleLabel": "Output",
            "type": "MANAGED_FOLDER",
            "$touched.it.roleLabel": true,
            "$invalid": false
         }
      ],
      "variablesEditionTile": {
         "behavior": "MODAL",
         "params": []
      },
      "runScenarioTile": {
         "scenarioId": "RUNE2E",
         "buttonText": "Run E2E"
      }
   },
   "instantiationPermission": "USE_APP_MASTER_PERMISSIONS",
   "limitedVisibilityEnabled": "ENABLED",
   "accessRequestsEnabled": "ENABLED",
   "allowedMissingConnections": [],
   "allowedMissingCodeEnvs": []
}

Summary:
Our original managed folder is getting replaced by folder of project, where the plugin is being used but we are not able to use that folder in original recipe.

> I hope there is a way to dynamically set the managed folder in custom recipes so that it could be taken up directly in 

[VitaliyD]

HI, 

If I understood your use case correctly, you need to read/write from/to a folder that is not specified as input/output in the custom Python recipe. You can use "ignore_flow=False" when creating a handle to a managed folder to be able to use the folder in the recipe without specifying the folder as input/output.

The only concern I have with this approach is that visually, in the flow, you won't see the usage of the folders that are not specified as input/output in this recipe.

Hope this helps.

Best,

Vitaliy