Instance wide Senario On/Off Swtich

tgb417

User Story:

As a Dataiku DSS instance Administrator that occasionally has to "turn off" Triggered Scenarios while working on a Dataiku DSS Instance. I would like a Master Triggered Scenario "Off switch". This would inactivate triggered Scenarios across all projects within a Single DSS Instance. This would mean that I can authoritatively know that Scenarios will not be updating my databases for example, or sending out notifications, or other things I might want to global suspend. And when I turn this master switch back "On" only the scenarios that were setup to to be running triggers will be reengaged to the configuration prior to using the "Off Switch". So this Off Process is reservable. When it comes to Dataiku DSS Maintenaince, this would allow me to more reliably and more reproducibly run my Dataiku DSS Instance.

Conditions of Satisfaction (COS):

• The Instance Scenario Off Switch is only available to Instance Administrators
• There is a visible indication on all individual project Scenario Screens that the System Administrator has turned off Scenarios Triggers for the entire Instance.
• Regardless of whether the Scenario was setup for automatic running. Individual Project Owners can not run any Scenarios when this off is set.
  • On the other had DSS Instance Administrators can override the Instance Scenario Off status at the individual Scenario. If the particular scenario that is selected is cascaded and calls other scenarios those scenarios will run for the Instance Administrators.
• Individual project Owners can not override Scenario off setting.
• When the Off Switch is chosen it warns the Administrator about the nature of the change
• When the Off Switch is chosen it gathers up all of the instance scenario states. (It will need these states when the master scenario "on switch" is selected.
• When Scenario On is selected the states of all Scenarios is placed back to where they were when the Off was set. (This can not really be done with Git because the Administrator may be making changes that they want to remember.)

Notes:

• From a UI point of view It might be nice to have this in the Automation Monitoring set of screens.
• I have an Instance with 36 or so Scenarios of which only 6-8 are setup to run automatically. The remainder of the Scenarios are called by these scenarios that have automatic triggers. Turning on the wrong one for automatic running could case problems, Particularly if there were not sequenced correctly.

Turribeach

I am guessing you know about the DSS Settings => Other => Misc => Disable automatic triggers setting. This setting sort of works partially as how you want but only works for Scenario schedules and any user can override it. Even if you were able to have full control of Scenarios as you want what will be your expectation with other workloads that can also affect your databases? For instance will individual jobs/recipes still be able to work? Will Dataiku API calls which could trigger recipes still work? What happens with Jupyter Notebooks and SQL Notebooks that can also perform DDLs? What happens with Macros? What about WebApps?

I think you will find that the only way to have 100% certainty that nothing is running in your Dataiku instance will be to Disable automatic triggers in the UI, disable login for any users and restart your DSS instance to clear any previous jobs.

Turribeach

Well well well, I shall be corrected. Turns out my understanding of how Dataiku and the "DSS Settings => Other => Misc => Disable automatic triggers" setting work was wrong (see here for more information).