Project Flow Creation w Code

cmjurs

If one has a particular type of project flow that gets used all the time within an organization, what is the best way to duplicate these common project structures to assure quality control. To accomplish this currently I am utilizing a "golden flow pattern" which I clone and rename to a new project manually as needed - but this seem clunky. Id like to get to more of a IAC type of paradigm for repeatability and management. Would it be possible to create an empty project flow strictly with code, creating empty datasets, recipes, automation schemes, etc (maybe using a plugin)? If it is possible, would this even be advisable? Why or why not?

If so, does anyone have an example they can share of doing something like this?

Thanks, CJ

---

Operating system used: Ubuntu

Alexandru

Hi @cmjurs
,
A plugin with a project creation macro comes to mind where you can of course include building datasets/recipe etc using the python APIs https://doc.dataiku.com/dss/latest/plugins/reference/project-creation-macros.html

You can look at Dataiku Applications or Application-as-Recipe.
https://doc.dataiku.com/dss/latest/applications/index.html
https://knowledge.dataiku.com/latest/mlops-o16n/dataiku-applications/tutorial-dataiku-application-as-recipe.html

Dataiku Applications are used to deliver Dataiku Solutions - https://doc.dataiku.com/dss/latest/applications/index.html to give an idea of what these projects could look like depending on your needs.

cmjurs

Thanks for your answer Alex. Ill check out the links you provided! ~CJ