Announcing the winners & finalists of the Dataiku Frontrunner Awards 2021! Read their inspiring stories

Is there a way to extract the test set of a Model to run comparative experiments ?

Solved!
Jérémie_BORDIER
Is there a way to extract the test set of a Model to run comparative experiments ?
I would like to score my own prediction method against the same test set that DSS generated to test a model. The test set being generated by a sampling method with randomization, it's quite tricky.

Is there a simple way to perform that ? Maybe by extracting the test set to a dataset on which I could do some analysis ?

Thanks !
0 Kudos
1 Solution
Clément_Stenac
Dataiker
Dataiker
Hi,

There are two solutions:

* Recommended: Split the dataset yourself and use the ability of the Analysis Models to use predefined train and test sets instead of letting it do a random split. At the moment, doing a random split using the split recipe is a bit tricky, you'd have to first create a random column with a Python processor in a preparation recipe

* Hackish / Not officially supported: When using memory-based models in DSS, the train and test sets are dumped as CSV files in the DSS datadir > analysis-data > project > analysis_id > model_id > splits

View solution in original post

2 Replies
Clément_Stenac
Dataiker
Dataiker
Hi,

There are two solutions:

* Recommended: Split the dataset yourself and use the ability of the Analysis Models to use predefined train and test sets instead of letting it do a random split. At the moment, doing a random split using the split recipe is a bit tricky, you'd have to first create a random column with a Python processor in a preparation recipe

* Hackish / Not officially supported: When using memory-based models in DSS, the train and test sets are dumped as CSV files in the DSS datadir > analysis-data > project > analysis_id > model_id > splits

View solution in original post

Mariak
Level 2

Hi,

i was wondering if the second solution is still available, since i would like to have the train and test set used in the model to proceed. And if yes, it'll be really helpful if you can provide the code needed for this.

Thank you!

0 Kudos
Labels (1)
A banner prompting to get Dataiku DSS