Survey banner
The Dataiku Community is moving to a new home! Some short term disruption starting next week: LEARN MORE

How to execute notbeooks in a scenario step

Ramya
Level 2
How to execute notbeooks in a scenario step

Im trying to run jupyter notebook using  Export notebook in Scenario but the scenario is failing.So is there any way to run the jupyter notebook in scenario??

0 Kudos
6 Replies
imanousar
Level 3
 
 

What is the error you getting if you try this?

0 Kudos
Ramya
Level 2
Author

I am currently working on utilizing pandas dataframes within my Jupyter notebook, where I execute various scripts to generate output for subsequent analysis. To streamline this process and reduce manual intervention, I am interested in exploring the possibility of automating the execution of the notebook using scenarios. Is there a way to achieve this? Your guidance and insights would be greatly appreciated.

0 Kudos
Turribeach

This is not the usual way of doing things. If you want to run some random Python code then add a Execute Python Code step to your scenario. If you want to refresh a dataset which is populated by a Python recipe then you add a Build step to your scenario, select the output dataset and Dataiku will automatically run th parent Python recipe to refresh your dataset. Otherwise can you please explain what are you trying to achieve as it is not clear from your question.

0 Kudos
Ramya
Level 2
Author

Thank you very much for sharing your input. I am currently working on utilizing pandas dataframes within my Jupyter notebook, where I execute various scripts to generate output for subsequent analysis. To streamline this process and reduce manual intervention, I am interested in exploring the possibility of automating the execution of the notebook using scenarios. Is there a way to achieve this? Your guidance and insights would be greatly appreciated.

0 Kudos
Turribeach

You should convert your Notebook to a recipe (click + CREATE RECIPE), persist your outputs as datasets and use a build scenario step like I mentioned before. You can read more about Scenarios below:

https://knowledge.dataiku.com/latest/mlops-o16n/automation/concept-scenarios.html

 

0 Kudos
Ramya
Level 2
Author

In order to perform the necessary analysis in my Jupyter notebook, having access to intermediate results is crucial. Automating the execution of the notebook would greatly simplify this process and enhance efficiency. It wil be helpful if  you could offer any insights on achieving this automation.

0 Kudos