Spark Recipe Error - Failed to write to output folder

Hi,

I am creating a Spark recipe. I have connected the relevant tables to the input and output of recipe. But when I run the recipe (complex), it failed while writing the output to one table (the recipe successfully wrote to other output tables). Interestingly when I try to re-run the recipe without making any modifications, It worked! And I also tested the code in Notebook and there is no issue at all. Why I am randomly getting this error?

Regards,

Operating system used: linux