Convert Visual Recipes into SQL steps using the Dataiku python API

Level 1
Convert Visual Recipes into SQL steps using the Dataiku python API


I am trying to convert all the Visual Recipes in a DSS project to SQL steps. I know that is possible in the web UI thanks to the convert to SQL recipe function. However, I would like to do it programmatically for several flows using the python API. Is there a way to do it?

Screenshot 2021-12-06 at 18.16.44.png

0 Kudos
1 Reply
Developer Advocate

Hi Andrea,

There is no endpoint in our public API to programmatically transform visual recipes into SQL ones in a straightforward way. 

One possible workaround would be to:

1. extract the SQL query from the visual recipe by using the Recipe public API  :

import dataiku
client = dataiku.api_client()
project = client.get_project("YOUR-PROJECT-KEY")
recipe = project.get_recipe("YOUR-RECIPE-ID")
sql = recipe.get_status().data.get("sql")

2. still with the Recipe public API, programmatically create a SQL recipe with that query and replace the visual recipe by remapping its inputs and outputs to the SQL recipe.

That being said, it is definitely not trivial to implement at scale for many projects, and it may not cover some edges, that's why I would instead recommend the slow-yet-safe visual way.

Hope this helps!





0 Kudos