Clearer visual experience when working with Partitioned Visual Model summary screens

0 Kudos

User Story

As a new Dataiku DSS user getting familiar with building partitioned visual models, I would like it to be clearer that there is a set of summary graphics for each of the partitions.  This would make using the partition feature easier and build confidence in users.  Making this feature feel less like an "easter egg" and more like a productive feature.

As a creator of partitioned visual models trying to use best practices of k-fold cross-validation testing, it would be helpful to see the (ยฑ 0.###) on the summary screens as we do with non-partitioned models.  This would allow for a quicker comparison of models.

Notes:

  1. No indication when you look at the screen you can click on anything.
  2. Hover over "All Partitions" no underline indication that you can click on the word "All Partitions"
  3. Even though you do not have a clue click on All Partitions and you get a graph similar to but not the same as the documentation here
  4. Click on the All Partitions "trained link" and you are taken to a training session log.
  5. Click on the name of the 1st partition in this case Florida. Or second portion name California in this case you get a different behavior and are taken to the partition Model summary page.
  6. Click on the Florida or California trained link and you are taken to a training session log.
  7. However, there appears to be nothing to click on to get the behavior described in the training session.  One end up having to click on the gray space between values on this summary screen to re-produce the feature shown in the training.

DSS Screen showing Partitioned visual model summery report.  There is a text cloud superimposed over the image that says "No indication here that you can click on anything."DSS Screen showing Partitioned visual model summery report. There is a text cloud superimposed over the image that says "No indication here that you can click on anything."

 

What I was trying to produce was the following as shown in the training materials.

Dataiku DSS model summary Screen showing a partition model Error and Actual values plot. There is a cartoon speach bubble added that says "I ended up clicking about here to get the Error and Actual values graph to show up." With it's pointer pointing to a space between the  name of the partition  in this case Florida and the word  "Trained" this location has been called out with a bit of transparent purple.Dataiku DSS model summary Screen showing a partition model Error and Actual values plot. There is a cartoon speach bubble added that says "I ended up clicking about here to get the Error and Actual values graph to show up." With it's pointer pointing to a space between the name of the partition in this case Florida and the word "Trained" this location has been called out with a bit of transparent purple.

 

For the second comment about k-fold cross validation here is what I'm currently seeing.

Model Summary screen from partitioned models, on the left side of the screen there is an indication of where cross validation results are shown on non-prartitioned models.  There is an overlaid speech bubble with the comment " It would be nice if Partitioned Models included the k-Fold cross validation results."  This bubble points to a place where these values would be helpful.Model Summary screen from partitioned models, on the left side of the screen there is an indication of where cross validation results are shown on non-prartitioned models. There is an overlaid speech bubble with the comment " It would be nice if Partitioned Models included the k-Fold cross validation results." This bubble points to a place where these values would be helpful.

 

For this testing, I'm using DSS V 8.0.4.

cc: @AshleyW 

--Tom
7 Comments
Krishna
Dataiker
 
Status changed to: In the Backlog
 

@Krishna ,

I've updated this idea with a second closely related idea about the display of k-fold cross-validation results in a partitioned model scenario.  I can break this out into a separate idea if you need it.

--Tom

@Krishna ,

I've updated this idea with a second closely related idea about the display of k-fold cross-validation results in a partitioned model scenario.  I can break this out into a separate idea if you need it.

I did find a way to see the k-fold cross-validation results.  However, it takes a fair amount of digging to find this information.

DSS partitioned visual model results.  On this screen is added a speach bubble pointing to both the Detail metrics menu item on the left and on the right the highlighted results of k-fold cross validation testing.  The speech bubble  says the following "I did eventually find the k-fold cross validating testing results on the โ€œDetailed metric summery screen.  However, it took some digging to find this and inter model comparison are difficult this way."DSS partitioned visual model results. On this screen is added a speach bubble pointing to both the Detail metrics menu item on the left and on the right the highlighted results of k-fold cross validation testing. The speech bubble says the following "I did eventually find the k-fold cross validating testing results on the โ€œDetailed metric summery screen. However, it took some digging to find this and inter model comparison are difficult this way."

--Tom

I did find a way to see the k-fold cross-validation results.  However, it takes a fair amount of digging to find this information.

DSS partitioned visual model results.  On this screen is added a speach bubble pointing to both the Detail metrics menu item on the left and on the right the highlighted results of k-fold cross validation testing.  The speech bubble  says the following "I did eventually find the k-fold cross validating testing results on the โ€œDetailed metric summery screen.  However, it took some digging to find this and inter model comparison are difficult this way."DSS partitioned visual model results. On this screen is added a speach bubble pointing to both the Detail metrics menu item on the left and on the right the highlighted results of k-fold cross validation testing. The speech bubble says the following "I did eventually find the k-fold cross validating testing results on the โ€œDetailed metric summery screen. However, it took some digging to find this and inter model comparison are difficult this way."

Krishna
Dataiker
 
Status changed to: Parked
 
Krishna
Dataiker

Thanks Tom for this very thorough request; the detailed steps for reproducibility are much appreciated.

On your first suggestion, I'm afraid I wasn't able to reproduce with safari or chrome (I tried on my existing install of 8.0.2, and upgraded to 8.0.4 as you specified) -- on hover, the rows should appear highlighted, indicating a clickable action (hopefully the gif below demonstrates).GIF of mouse hover over partition recordsGIF of mouse hover over partition records

 

That being said, I acknowledge the suggestion to make it even more explicit, and have logged it along with the second suggestion around displaying the k-fold margin on the model summary screen.

Status changed to: In Backlog

Thanks Tom for this very thorough request; the detailed steps for reproducibility are much appreciated.

On your first suggestion, I'm afraid I wasn't able to reproduce with safari or chrome (I tried on my existing install of 8.0.2, and upgraded to 8.0.4 as you specified) -- on hover, the rows should appear highlighted, indicating a clickable action (hopefully the gif below demonstrates).GIF of mouse hover over partition recordsGIF of mouse hover over partition records

 

That being said, I acknowledge the suggestion to make it even more explicit, and have logged it along with the second suggestion around displaying the k-fold margin on the model summary screen.

Yes, it does work. 

However, you know that the graphics are there.  As a new user, I did not.  The only indication is that the mouse pointer changes and yes the rows highlight.  The issue for me was more around, in most other places in DSS when you have something clickable you end up with an underline under something.  These items do not end up with the same behavior.  (And because of all of the graphic building, the long delay to open these rows also felt really odd.)

Regarding including the error range in the overview of all of the summaries is that something that is going to be doable?

--Tom

Yes, it does work. 

However, you know that the graphics are there.  As a new user, I did not.  The only indication is that the mouse pointer changes and yes the rows highlight.  The issue for me was more around, in most other places in DSS when you have something clickable you end up with an underline under something.  These items do not end up with the same behavior.  (And because of all of the graphic building, the long delay to open these rows also felt really odd.)

Regarding including the error range in the overview of all of the summaries is that something that is going to be doable?

MichaelG
Community Manager
Community Manager
 
I hope I helped! Do you Know that if I was Useful to you or Did something Outstanding you can Show your appreciation by giving me a KUDOS?

Looking for more resources to help you use DSS effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as โ€˜Accepted Solutionโ€™ to help others like you!
Status changed to: In the Backlog