Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I have a python recipe that points to a folder on a GCP cloud storage connection. Overall the recipe works well, however when the folder is empty, because I clean it periodically, the recipe fails at the validation step:
Validation failed: Failed to compute recipe status: Folder doesn't exist
I have checked on the GCP console and I can confirm that the folder was indeed removed when what I wanted was to simply purge the files that were inside the folder. The action that I performed was a "Clear" on the folder.
I would like to be able to run this recipe without this type of error, either by being able to clear the contents of the folder without it being deleted or by being able to skip the validation made by DSS before the recipe is run. I tried overrunning the params.skipPrerunValidate recipe variable and setting it to true, but this does not seem to work. Does anyone have any other ideas?
Hi @Mario_Burbano ,
I wanted to confirm if you are writing to a managed folder that points to GCS? If so, it is the case that clicking “Clear” on a managed folder via the UI will clear the contents of the managed folder, including the “prefix” itself.
However, I would expect the prefix to get re-created when you run your recipe. If you are able to reproduce the error, would you mind attaching the following afterwards?
Then we can see what options might be available to avoid the issue you are facing.
Hi @Mario_Burbano ,
Thank you for attaching this information! I haven't quite been able to reproduce the scenario that you seem to run into here. Would you mind opening a support ticket and attaching a job diagnostic of the job run that fails? You can get the diagnostic from the job page, by clicking on Actions > Download job diagnosis. I think that will make this easier to look into.