I think it would be very useful to be able to move objects and flow zones around in the flow display. It appears Dataiku determines where each recipe, dataset, etc go in the flow and I cannot edit that. I have used Alteryx in the past and it had that ability, which I liked. It allows me to organize the flow however I see fit and make it more visually appealing and more understandable for someone looking at it.
Hi @Ben_Rutan ,
Thanks for the feedback 🙂
The solution to managing large flows is by using flow zones. Through flow zones, you can split your flow into small, named/colored zones, which makes it much easier to orient yourself and focus only where you need to, especially when working with a very large flow.
Have you worked with flow zones/found them helpful in managing large flows? Happy to collect any other feedback here, and I will log your request, as I know that flow zones wasn't exactly what you were looking for, but is a solution that many users have found helpful 🙂
Hi @ktgross15 ,
I have used flow zones and they are somewhat helpful, but I think it would be more beneficial if I could rearrange each recipe, dataset, etc however I see fit instead of how Dataiku decides to display them. Just something to add to the wish list 🙂
In my experience Dataiku does a great job it drawing the flow in a reasonable way. There are indeed cases where things do look a bit awkward (see screen shot below) but even in this situation I don't think I will spend the time it would take to reorg the flow. And any manual flow reorg would be subject to manual maintenance as well, meaning new flow objects will need to be placed in sensible places so I don't think a lot of people will be willing to invest their time on this. Just look at the mess that are most people's Windows desktop or the iPhone App home page.
Agreed! Moving and nesting flow zones - organizing components more effectively in sub sections in a flow zone.
Hi @Ben_Rutan ,
Thanks for this suggestion! This idea it not currently planned, but we’ll circle back to this thread if we’re looking for more info in the future.
The Flow is the visual representation of how datasets, recipes, and models work together to move data through an analytical pipeline. Projects can quickly become complex with hundreds of objects on the Flow. We sometimes call these ‘spaghetti’ flows 🙂
As it grows in complexity, manually positioning objects and zones contributes to an unnecessary burden to maintain this arrangement over time, as @Turribeach mentioned. We’ve also noticed that manual positioning is very personal: while it may facilitate one person’s understanding of a project, it often does the opposite for others.
To help with organizing objects in a complex project, I’ll second @ktgross15 ’s suggestion to use zones. Since Dataiku projects are built around the notion of collaboration, we find that a layout structured around flow zones is more neutral than moving things around.
At Dataiku, we use zones with great success to arrange datasets/recipes/models into logical groupings, making large flows much easier to manage and collaborate on. They’re automatically laid out for me, and I never worry about them colliding or needing to resize them. With a description in the R pane, it’s easy for me (and others) to understand what a zone contributes to the project and see how all of the pieces of the pipeline fit together. (Yes, it drives me nuts when objects move; I've found zones help with that too)
As a bonus, check out the ‘Flow Zones’ flow view of a project. Clicking the ‘hide zones’ button can be a surprising reminder of how much they—and anchoring datasets—can do for you
+1 for this idea regarding flow zones (can't add a thumbs up as it has been deactivated in this thread).
Though dataiku handles positioning very well, sometimes it can be improved and it would be nice if the user could customize things when explicitly asked.
I am especially thinking of flow zones. Here is an example where dataiku's position is not intuitive:
The building order follows the numbers. So in this situation the 3rd flow zone appears to the left of the 2nd flow zone, which doesn't make much sense.
Only members of the Community can comment.