How can I use spark MLLib in a custom model ?

HugoDupeux Registered Posts: 3 ✭✭✭✭

Good morning,

I want to create a custom model using MLLib (because I need to use Frequent Pattern Mining algorithms who are already in there).

I found some documentation on how create a custom model :

But none of them explain how to do it (in my comprehension). Python and sklearn are well explained and shown but not MLLib. I wonder if there is a code example I could use to help me or a more sepcific documentation.

I am using dataiku V7, not 8. I don't know if there is an impact on custom model.

Thank you.

Kind Regards,



  • Triveni
    Triveni Dataiker, Registered Posts: 20 Dataiker
    edited July 17


    Inside of the custom model editor there is a sample provided, I have pasted it below.

    // This sample code uses a standard MLlib algorithm, the RandomForestRegressor.
    // import the Estimator from
    // instantiate the Estimator
    new RandomForestRegressor()
       .setLabelCol("ot_avg")  // Must be the target column
       .setFeaturesCol("__dku_features")  // Must always be __dku_features
       .setPredictionCol("prediction")  // Must always be prediction
Setup Info
      Help me…