Irrelevant warnings during import or Automation node deployment

We have lots of plugins on our Designer instance since we are testing them to see if we can use them in our projects. However this means that whenever we deploy a project to the Automation node we get warnings on deployment as the plugin list in the Automation node does not match with the Designer node. This is a bad user experience. Ideally Dataiku should be able to figure out which plugins are used in each project and don't throw Warnings for plugins that are missing in the Automation node but actually are not used in the project being deployed. Dataiku is already able to determine which connections and code environments are used in a project and validate these exist in the destination Automation node being deployed to so the same should be done for Plugins. 

(Topic title edited by moderator to be more descriptive. Original title "Warnings during import or Automation node")

15 Comments

User Story:

As an Analyst given a project from another community member, I would like the warning during importation to be more targeted to the actual project I'm installing, so that I can more confidently exchange and receive projects from other community members.

Notes:

Here are the types of warnings that one can receive.  This seems like it would be intimidating to a New Analyst on the system.

Import Error.jpg

 

As the project, I was importing did not have any time series work in it.  This warning was irrelevant.  As I was not going to continue development on this project the warning about the project-developer plugin was also irrelevant to me as an Analyst.  That said these warnings should be available to more advanced users.

Here is a link to a community post that details an example of these challenges for an analyst to import a project.

https://community.dataiku.com/t5/Online-Events/Converting-your-Dataiku-DSS-Project-into-a-Reusable-A... 

--Tom

User Story:

As an Analyst given a project from another community member, I would like the warning during importation to be more targeted to the actual project I'm installing, so that I can more confidently exchange and receive projects from other community members.

Notes:

Here are the types of warnings that one can receive.  This seems like it would be intimidating to a New Analyst on the system.

Import Error.jpg

 

As the project, I was importing did not have any time series work in it.  This warning was irrelevant.  As I was not going to continue development on this project the warning about the project-developer plugin was also irrelevant to me as an Analyst.  That said these warnings should be available to more advanced users.

Here is a link to a community post that details an example of these challenges for an analyst to import a project.

https://community.dataiku.com/t5/Online-Events/Converting-your-Dataiku-DSS-Project-into-a-Reusable-A... 

fsergot
Dataiker

Good day,

We have recorded this in the product backlog. We will let you know any progress.

As a side note, having a 100% certainty on which plugins are actually used is not trivial at all, and we don't want to give a false sense of confidence.

Status changed to: In Backlog

Good day,

We have recorded this in the product backlog. We will let you know any progress.

As a side note, having a 100% certainty on which plugins are actually used is not trivial at all, and we don't want to give a false sense of confidence.

allow me to illustrate this annoying alert 😇

screenshot-warnings.jpg

 

allow me to illustrate this annoying alert 😇

screenshot-warnings.jpg

 

"As a side note, having a 100% certainty on which plugins are actually used is not trivial at all, and we don't want to give a false sense of confidence." => Understood but this is not by design, so it can easily be changed. For instance Dataiku could require developers to enable plugins in the Project Settings before being able to use them, which would create a clear way to determine which plugins are "enabled" in each project. Dataiku could also maintain a registry of recipe plugins per project, so this wouldn't need to hard to find out. 

"As a side note, having a 100% certainty on which plugins are actually used is not trivial at all, and we don't want to give a false sense of confidence." => Understood but this is not by design, so it can easily be changed. For instance Dataiku could require developers to enable plugins in the Project Settings before being able to use them, which would create a clear way to determine which plugins are "enabled" in each project. Dataiku could also maintain a registry of recipe plugins per project, so this wouldn't need to hard to find out. 

fsergot
Dataiker

Sorry for the late update but this was added in our backlog. This would indeed facilitate the sharing of projects between instances and raise more relevant warnings.

Status changed to: In Backlog

Sorry for the late update but this was added in our backlog. This would indeed facilitate the sharing of projects between instances and raise more relevant warnings.

fsergot
Dataiker

@Turribeach , that would indeed be an option. Note however that it would require more work on the product side to add a feature to activate a plugin by project, not counting the impact on existing usage.

I have added the idea on the original request in our backlog!

As a side note, we hope to ship a workaround for this on bundle deployment in a coming version with the ability to disable this deployment warning for non-admin users (who can't do much about it) but still keep it in the logs & for the admin.

@Turribeach , that would indeed be an option. Note however that it would require more work on the product side to add a feature to activate a plugin by project, not counting the impact on existing usage.

I have added the idea on the original request in our backlog!

As a side note, we hope to ship a workaround for this on bundle deployment in a coming version with the ability to disable this deployment warning for non-admin users (who can't do much about it) but still keep it in the logs & for the admin.

Ummmm I guess that comes from this idea which is just trying to hide the problem away. If I am honest I will say that I don't really like the solution. As an Administrator what do I do? Do I enable the "disable this deployment warning for non-admin users" option so my users don't see what are most likely false alerts when deploying projects to the Automation Node but risk missing an important dependancy is some cases causing a failed pipeline run in Production? It's a tough choice and one that we will probably error on the side of caution so we will not be using this new feature. I understand that fixing this properly will take more work but it's the right thing to do when you consider that all other external project artifacts like connections, code environments, etc are traceable within the project itself. So plugins should be no different.

If you gave the choice of having the "disable this deployment warning for non-admin users" option or an option of knowing when plugins are used 99% (*) of the time (as you said it's hard to have a 100% certainty) then I will take the latter every day, as I can then deal with the 1% as an exception. The 99% option will still be much safer than the "disable this deployment warning for non-admin users" option which will basically cause failures for every project that uses a plugin until the plugin is installed.

(*) Obviously I don't know exactly in what scenarios it's difficult to know a plugin is used but I presume these are edge cases and the majority of the use cases are covered.

Ummmm I guess that comes from this idea which is just trying to hide the problem away. If I am honest I will say that I don't really like the solution. As an Administrator what do I do? Do I enable the "disable this deployment warning for non-admin users" option so my users don't see what are most likely false alerts when deploying projects to the Automation Node but risk missing an important dependancy is some cases causing a failed pipeline run in Production? It's a tough choice and one that we will probably error on the side of caution so we will not be using this new feature. I understand that fixing this properly will take more work but it's the right thing to do when you consider that all other external project artifacts like connections, code environments, etc are traceable within the project itself. So plugins should be no different.

If you gave the choice of having the "disable this deployment warning for non-admin users" option or an option of knowing when plugins are used 99% (*) of the time (as you said it's hard to have a 100% certainty) then I will take the latter every day, as I can then deal with the 1% as an exception. The 99% option will still be much safer than the "disable this deployment warning for non-admin users" option which will basically cause failures for every project that uses a plugin until the plugin is installed.

(*) Obviously I don't know exactly in what scenarios it's difficult to know a plugin is used but I presume these are edge cases and the majority of the use cases are covered.

ClaudiusH
Dataiker Alumni

FYI: We have merged the idea earlier submitted by Tom which was referenced in the comments into this one as the solution to be worked out should tackle both. 

FYI: We have merged the idea earlier submitted by Tom which was referenced in the comments into this one as the solution to be worked out should tackle both. 

@ClaudiusH ,

This post is actually a subset of my original post.  The problem that I was pointing to is I was getting errors about components that were not used in my project. Plug-ins were only one part. There were confusing bits about connections and other stuff. My original product suggestion was partially about don’t tell me things about components that have not relationship to the current project. And tell me these things in such a way that an analyst not fully conversant in the total architecture might be successful in importing a project from one design node to the next.  Versions since 2020 have made some helpful changes. However, my comments are much broader than just plugins.  

--Tom

@ClaudiusH ,

This post is actually a subset of my original post.  The problem that I was pointing to is I was getting errors about components that were not used in my project. Plug-ins were only one part. There were confusing bits about connections and other stuff. My original product suggestion was partially about don’t tell me things about components that have not relationship to the current project. And tell me these things in such a way that an analyst not fully conversant in the total architecture might be successful in importing a project from one design node to the next.  Versions since 2020 have made some helpful changes. However, my comments are much broader than just plugins.