Survey banner
The Dataiku Community is moving to a new home! New posts are now disabled and the community will shortly be in temporary read only mode: LEARN MORE

Project Flow Creation w Code

Solved!
cmjurs
Level 3
Project Flow Creation w Code

If one has a particular type of project flow that gets used all the time within an organization, what is the best way to duplicate these common project structures to assure quality control. To accomplish this currently I am utilizing a "golden flow pattern" which I clone and rename to a new project manually as needed - but this seem clunky. Id like to get to more of a IAC type of paradigm for repeatability and management. Would it be possible to create an empty project flow strictly with code, creating empty datasets, recipes, automation schemes, etc (maybe using a plugin)? If it is possible, would this even be advisable? Why or why not?

If so, does anyone have an example they can share of doing something like this?

Thanks, CJ


Operating system used: Ubuntu

0 Kudos
1 Solution
AlexT
Dataiker

Hi @cmjurs ,
A plugin with a project creation macro comes to mind where you can of course include building datasets/recipe etc using the python APIs  https://doc.dataiku.com/dss/latest/plugins/reference/project-creation-macros.html

You can look at Dataiku Applications or Application-as-Recipe. 
https://doc.dataiku.com/dss/latest/applications/index.html
https://knowledge.dataiku.com/latest/mlops-o16n/dataiku-applications/tutorial-dataiku-application-as...

Dataiku Applications are used to deliver Dataiku Solutions  - https://doc.dataiku.com/dss/latest/applications/index.html to give an idea of what these projects could look like depending on your needs. 

View solution in original post

2 Replies
AlexT
Dataiker

Hi @cmjurs ,
A plugin with a project creation macro comes to mind where you can of course include building datasets/recipe etc using the python APIs  https://doc.dataiku.com/dss/latest/plugins/reference/project-creation-macros.html

You can look at Dataiku Applications or Application-as-Recipe. 
https://doc.dataiku.com/dss/latest/applications/index.html
https://knowledge.dataiku.com/latest/mlops-o16n/dataiku-applications/tutorial-dataiku-application-as...

Dataiku Applications are used to deliver Dataiku Solutions  - https://doc.dataiku.com/dss/latest/applications/index.html to give an idea of what these projects could look like depending on your needs. 

cmjurs
Level 3
Author

Thanks for your answer Alex. Ill check out the links you provided! ~CJ

0 Kudos