Survey banner
The Dataiku Community is moving to a new home! We are temporary in read only mode: LEARN MORE

dataset.build() doesn't work for streaming python recipe output dataset

Solved!
Sangavi_M
Level 2
dataset.build() doesn't work for streaming python recipe output dataset

I was trying to start a job in order to build the dataset, whose recipe parent is a very simple streaming python recipie. It doesn't throw any error but doesnt build the dataset as well

But same works fine with a normal python recipe and I'm able to build a dataset.

is there any way i can achieve the same for a streaming python recipie? This is what I was doing:

client = dataiku.api_client()
project = client.get_default_project()


sample_df = project.get_dataset("sample")
job = sample_df.build()
print(job)

0 Kudos
1 Solution
Turribeach

Continuous recipes are a different beast and you need to use special methods to start and stop them:

import dataiku

client = dataiku.api_client()
project = client.get_project('some project key with continuous activities')
continuous_activities = project.list_continuous_activities()

for recipe in continuous_activities:
    recipe_running = str(recipe.get_status()['mainLoopState']['futureInfo']['alive'])
    print(str(recipe.recipe_id) + " - Running: " + recipe_running)
    if recipe_running == "False":
        recipe.start()

 

View solution in original post

2 Replies
Turribeach

Continuous recipes are a different beast and you need to use special methods to start and stop them:

import dataiku

client = dataiku.api_client()
project = client.get_project('some project key with continuous activities')
continuous_activities = project.list_continuous_activities()

for recipe in continuous_activities:
    recipe_running = str(recipe.get_status()['mainLoopState']['futureInfo']['alive'])
    print(str(recipe.recipe_id) + " - Running: " + recipe_running)
    if recipe_running == "False":
        recipe.start()

 

Sangavi_M
Level 2
Author

Thank you for the prompt response, this works for me!

0 Kudos