I trying to create project ,where I can recalculate my data based the user input . below is my flow in dataiku project
Data - this data set ,i have uploaded through csv file
webapp_input - This dataset ,I am loading from webapp (created in html,js and python backend )
Output- this data set I am creating using python recipe
Problem statement :
whenever ,I am adding more data in webapp_input through dataiku webapp .I have to run the python receipt manually to re-build the "output"dataset .
But I am trying to trigger it through webapp itself .As soon as will update the data in UI .it should store in webapp_input dataset (which is working fine ) and re-build the "output" dataset .
I have tried following python scripts for this .
import pandas as pd
import numpy as np
import dataiku
import json
import dataikuapi
client = dataiku.api_client()
def jobrun():
project = client.get_project('Testing')
definition = {
"type" : "NON_RECURSIVE_FORCED_BUILD",
"outputs" : [{
"id" : "output",
"partition" : "NP"
}]
}
job = project.start_job(definition)
state = ''
while state != 'DONE' and state != 'FAILED' and state != 'ABORTED':
time.sleep(1)
state = job.get_status()['baseStatus']['state']
Even though the dataset is present in the Project, it’s giving us below error message.
DataikuException: java.lang.IllegalArgumentException: Dataset not found or not buildable: Testing.output
I have tried second option :
from dataiku.scenario import Scenario
client = dataiku.api_client()
scenario.build_dataset("output ")
Non of the options are working .would you be to help me with solving this issue .or any other approach for my requirement .
Thanks