Background and Introduction:
Dataiku DSS has many different instance types whether it be Dataiku DSS Designer Nodes, Automation Nodes, API Deployer etc. It feels as if there may be a node missing from the Dataiku DSS platform. Specifically one that can be used to help deploy web apps and test out plugins. A place where folks are able to separate their development environment from the designer node which is really considered "production" for plugin developers and web app developers. Since the goal of this instance type would be to cater to the development that happens "behind-the-scenes" for the DSS Designer Node, "Backstage" may be an appropriate name.
Additionally, having a node that caters to plugin development and web app development may also help separate out some of the features of the DSS project page in designer nodes which can sometimes be overwhelming for new DSS users when they try to navigate through the top navigation bar with all of the tools they can use in their project. Perhaps also having a designer node that is more minimal may help on the UX side as well.
Description (User Story):
As a Dataiku DSS developer I want a new DSS instance type that caters to plugin and web app development so that I have an easier time as a developer versioning, testing and deploying my web apps and plugins to Dataiku DSS designer instances which may act as my "Production environment" but are sort of used today like a development/production all-in-one.
Example of Enhancements:
I'll know it is successful when the following features/enhancements are implemented for the DSS instance version:
Medium - High
Note, I realize this will most likely take a while to build.
The overall Dataiku DSS experience for both the developers and end users will be greatly enhanced. The reason for this is because if the plugins or web apps themselves are not properly developed by the DSS developer persona (have bugs) then the overall image and experience of Dataiku is not properly reflected since these plugins and web apps help extend the functionality of DSS and what it has to offer. If there is anyway Dataiku can help encourage quality control and best practices then it is a win for everyone.
Like I mentioned before in the background section, this feature may help decouple the DSS project page which is sometimes overwhelming for DSS users especially new ones. Separating out things like plugin development, web app development may help alleviate this as well as provide an opportunity for these features to grow and be built upon.
Enhancing this experience encourages end users to develop well tested and robust plugins and web apps that they can share with the larger dataiku community.
Additionally, I will highlight some of the pain points experienced today to give you a sense of why this may be something that is needed. Perhaps shedding light on these pain points may help us construct any alternative solutions if necessary.
What are some of the pain points with creating plugins and web apps in DSS?
SSH keys for security groups so that developers can push and pull to remote git hosting services isn't something that is easy to configure on the DSS server out of the box. Making it hard to utilize services like GitHub, Bit Bucket, or GitLab through DSS.
When developing plugins and web apps in the designer nodes today it feels as if we are following bad practices by not separating the development environment from our production environment since there is no deployment process built into the development of plugins or web apps. You simply just create it in the designer node from beginning to end.
There is no part in the DSS GUI that enforces or recommends testing of your plugin or web app or perhaps even offer off-the shelf standard tests (scenarios) to help speed up the development processes via standard flow quality control checking.
It is hard to collaborate with other developers on a web app since there is no way to work on different branches from each other (I believe....maybe branches for the overall project can do this??) slowing down the development process.
Let me know if there is anything you would like me to clarify or any ideas you would like to add yourself to this.
Attached are some of my ideas expressed visually via PDF and PowerPoint. This shows a rough draft skeleton.