Running your ml flow from bokeh web-app submit button in Dataiku

rajat_malik

Hello everyone!

So, i am working on a project where we have a webApp in bokeh and ml pipeline in flow section of DataIku. So, the application has several input fields which should be supplied to the ml-pipeline residing in the flow. Now, once the user press submit button in bokeh webapp, the ml-pipeline in flow should start running with the input provided from bokeh app. How can i make it happend ?

Also, is there a way i can load the latest trained model from flow in bokeh web app directly without using API ?

pmasiphelps

Hi,

Here's some python code using the API that assumes you have a flow starting with an "Uploaded File" type dataset. This assumes you've written the code to read in an uploaded file from the webapp user - then it places this file in the initial dataset in your flow.

import dataiku
from dataiku import pandasutils as pdu
import pandas as pd

client = dataiku.api_client()

proj = client.get_project("PROJECT_KEY")

ds = proj.get_dataset("UPLOADED_FILE_DATASET_NAME")

#clear any existing file from this dataset
ds.clear()

upload_file_path = "your_path_here"

with open(upload_file_path, "rb") as f:
    ds.uploaded_add_file(f, upload_file_path)

Then, assuming you've created a scenario in your project that runs your ML pipeline, you can run this scenario via another API call in your webapp.

scenario = proj.get_scenario("SCENARIO_ID")

trigger_fire = scenario.run()

There are a number of variants to running scenarios via the API - doc here: https://doc.dataiku.com/dss/latest/python-api/scenarios.html#run-a-scenario

If using Bokeh, the way to load the latest trained model from the flow would indeed be via the API.

If you're interested in developing applications using flow components (replacing files in datasets, running scenarios, pulling ML model info) without having to code them yourself using the APIs - project applications would be a great thing to check out. Here's some hands-on tutorials: https://academy.dataiku.com/dataiku-applications-tutorials-open

Best,

Pat