How to execute notbeooks in a scenario step

[Ramya]

Im trying to run jupyter notebook using  Export notebook in Scenario but the scenario is failing.So is there any way to run the jupyter notebook in scenario??

[imanousar]

What is the error you getting if you try this?

[Ramya]

I am currently working on utilizing pandas dataframes within my Jupyter notebook, where I execute various scripts to generate output for subsequent analysis. To streamline this process and reduce manual intervention, I am interested in exploring the possibility of automating the execution of the notebook using scenarios. Is there a way to achieve this? Your guidance and insights would be greatly appreciated.

[Turribeach]

This is not the usual way of doing things. If you want to run some random Python code then add a Execute Python Code step to your scenario. If you want to refresh a dataset which is populated by a Python recipe then you add a Build step to your scenario, select the output dataset and Dataiku will automatically run th parent Python recipe to refresh your dataset. Otherwise can you please explain what are you trying to achieve as it is not clear from your question.

[Ramya]

Thank you very much for sharing your input. I am currently working on utilizing pandas dataframes within my Jupyter notebook, where I execute various scripts to generate output for subsequent analysis. To streamline this process and reduce manual intervention, I am interested in exploring the possibility of automating the execution of the notebook using scenarios. Is there a way to achieve this? Your guidance and insights would be greatly appreciated.

[Turribeach]

You should convert your Notebook to a recipe (click + CREATE RECIPE), persist your outputs as datasets and use a build scenario step like I mentioned before. You can read more about Scenarios below:

https://knowledge.dataiku.com/latest/mlops-o16n/automation/concept-scenarios.html

[Ramya]

In order to perform the necessary analysis in my Jupyter notebook, having access to intermediate results is crucial. Automating the execution of the notebook would greatly simplify this process and enhance efficiency. It wil be helpful if  you could offer any insights on achieving this automation.