How to execute notbeooks in a scenario step
Im trying to run jupyter notebook using Export notebook in Scenario but the scenario is failing.So is there any way to run the jupyter notebook in scenario??
Answers
-
Ioannis Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 28 ✭✭✭✭✭OptionsWhat is the error you getting if you try this?
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,757 NeuronOptionsThis is not the usual way of doing things. If you want to run some random Python code then add a Execute Python Code step to your scenario. If you want to refresh a dataset which is populated by a Python recipe then you add a Build step to your scenario, select the output dataset and Dataiku will automatically run th parent Python recipe to refresh your dataset. Otherwise can you please explain what are you trying to achieve as it is not clear from your question.
-
Ramya Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 8 ✭Options
Thank you very much for sharing your input. I am currently working on utilizing pandas dataframes within my Jupyter notebook, where I execute various scripts to generate output for subsequent analysis. To streamline this process and reduce manual intervention, I am interested in exploring the possibility of automating the execution of the notebook using scenarios. Is there a way to achieve this? Your guidance and insights would be greatly appreciated.
-
Ramya Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 8 ✭Options
I am currently working on utilizing pandas dataframes within my Jupyter notebook, where I execute various scripts to generate output for subsequent analysis. To streamline this process and reduce manual intervention, I am interested in exploring the possibility of automating the execution of the notebook using scenarios. Is there a way to achieve this? Your guidance and insights would be greatly appreciated.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,757 NeuronOptions
You should convert your Notebook to a recipe (click + CREATE RECIPE), persist your outputs as datasets and use a build scenario step like I mentioned before. You can read more about Scenarios below:
https://knowledge.dataiku.com/latest/mlops-o16n/automation/concept-scenarios.html
-
Ramya Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 8 ✭Options
In order to perform the necessary analysis in my Jupyter notebook, having access to intermediate results is crucial. Automating the execution of the notebook would greatly simplify this process and enhance efficiency. It wil be helpful if you could offer any insights on achieving this automation.