Is there a way to extract the test set of a Model to run comparative experiments ?

Jérémie_BORDIER Registered Posts: 3 ✭✭✭✭
I would like to score my own prediction method against the same test set that DSS generated to test a model. The test set being generated by a sampling method with randomization, it's quite tricky.

Is there a simple way to perform that ? Maybe by extracting the test set to a dataset on which I could do some analysis ?

Thanks !

Best Answer

  • Clément_Stenac
    Clément_Stenac Dataiker, Dataiku DSS Core Designer, Registered Posts: 753 Dataiker
    Answer ✓

    There are two solutions:

    * Recommended: Split the dataset yourself and use the ability of the Analysis Models to use predefined train and test sets instead of letting it do a random split. At the moment, doing a random split using the split recipe is a bit tricky, you'd have to first create a random column with a Python processor in a preparation recipe

    * Hackish / Not officially supported: When using memory-based models in DSS, the train and test sets are dumped as CSV files in the DSS datadir > analysis-data > project > analysis_id > model_id > splits


  • Mariak
    Mariak Registered Posts: 5 ✭✭✭✭


    i was wondering if the second solution is still available, since i would like to have the train and test set used in the model to proceed. And if yes, it'll be really helpful if you can provide the code needed for this.

    Thank you!

Setup Info
      Help me…