Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Flow zones and support for them in scenarios
Added on June 1, 2021 11:31AM
Likes: 2
Replies: 7
tomas_sobotik
Partner, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 6 Partner
Hello,
I am exploring flow zones possibilities and I am missing one feature. Is there planned to support flow zones in scenarios?
Instead of building single datasets I would like put there to build whole zone. What would be also nice is trigger another scenario when some flow zone has been refreshed.
My use case:
Having complex flow which consists of two main parts and producing several output datasets. Now I have two main scenarios which runs one after each other. I would like to achieve better parallelism to run the flows in parallel but there are parts which are common for both so those can't run in parallel.
Now going through the flow and tagging datasets as flowA or flowB or common and later will need to create a new scenarios to parallelise the processing. So I was thinking how to use flow zones for it but I ended up in scenarios where I still would need to define the build on dataset level not zone level.
Comments
-
CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭Hi @tomas_sobotik
thanks for reaching out as always. We felt this would be better suited on the Product Ideas board so we moved your post and officially made it an idea. Thanks again! -
Ashley Dataiker, Alpha Tester, Dataiku DSS Core Designer, Registered, Product Ideas Manager Posts: 165 DataikerHi @tomas_sobotik
, thank you for your idea!What you've mentioned looks like two different ideas:
- Be able to build everything in a given Flow Zone
- Be able to trigger scenario steps based on the completion of (1)
Does that sound about right? If so, would you mind splitting them into two different ideas?
Take care,
Ashely
-
tomas_sobotik Partner, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 6 Partner
Hi @AshleyW
,yes, you got it right, two ideas. So do you want me to create a second post and place the idea #2 there?
Cheers,
Tomas
-
Ashley Dataiker, Alpha Tester, Dataiku DSS Core Designer, Registered, Product Ideas Manager Posts: 165 Dataiker
Hi,
That would be great, thank you! You can mention that the second relates to the first since they're related capabilities.
Best,
Ashley
-
Hello @tomas_sobotik
@bhazelwood
and all future readers of this post!I'm pleased to share that building flow zones has been released as part of DSS V12; see detail in our release notes and knowledge base article.
Note that building flow zones as part of scenarios is not part of V12, but is also something we plan to implement in the short term (more updates to come on that later).
Let us know if you have any questions in the interim!
Katie
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,315 Neuron"Note that building flow zones as part of scenarios is not part of V12, but is also something we plan to implement in the short term (more updates to come on that later)."
Oh yes please we would love this too! As a work around can this be done via the API? ie Can I use Python code in a Scenario to rebuild a Flow Zone?
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,315 Neuron
Ummm sad to see this is still outstanding and that this idea has set to "released" when there is still no "proper" support for building flow zone in Scenarios. As such I have raised a new Product Idea which I hope it's quickly implemented since this functionality has been released almost a year ago and we still don't have it available in Scenarios:
