Create Python APIs for Copy Action and Copy Subflow Action
Turribeach
Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,090 Neuron
Our objective is create an automated tool to do regression testing on parts of the flow. We don't want to do full flow testing since it will take a significant amount of time to duplicate our target projects and their target dependencies. Also testing whole project will be much more compute work than we want to allocate for this task. We therefore want to select some critical recipes in some critical flows and regression test them in a different environment before and after applying changes to the environment.
We looked at options for duplicating flows. In DSS there is a Copy action (which allows multi selected items) and Flow Action => Copy Subflow which clones a flow zone. However neither of these are available in the Dataiku API. So this idea is to enable those features in the API to be able to use them programmatically. While it is possible to create recipes and datasets using the Dataiku API this is not a trivial task as you have to really duplicate all the properties/methods to get a true copy of an object.