Is there a way to extract the test set of a Model to run comparative experiments ?

Options
Jérémie_BORDIER
Jérémie_BORDIER Registered Posts: 3 ✭✭✭✭
I would like to score my own prediction method against the same test set that DSS generated to test a model. The test set being generated by a sampling method with randomization, it's quite tricky.

Is there a simple way to perform that ? Maybe by extracting the test set to a dataset on which I could do some analysis ?

Thanks !

Best Answer

  • Clément_Stenac
    Clément_Stenac Dataiker, Dataiku DSS Core Designer, Registered Posts: 753 Dataiker
    Answer ✓
    Options
    Hi,

    There are two solutions:

    * Recommended: Split the dataset yourself and use the ability of the Analysis Models to use predefined train and test sets instead of letting it do a random split. At the moment, doing a random split using the split recipe is a bit tricky, you'd have to first create a random column with a Python processor in a preparation recipe

    * Hackish / Not officially supported: When using memory-based models in DSS, the train and test sets are dumped as CSV files in the DSS datadir > analysis-data > project > analysis_id > model_id > splits

Answers

  • Mariak
    Mariak Registered Posts: 5 ✭✭✭✭
    Options

    Hi,

    i was wondering if the second solution is still available, since i would like to have the train and test set used in the model to proceed. And if yes, it'll be really helpful if you can provide the code needed for this.

    Thank you!

Setup Info
    Tags
      Help me…