Model Scoring takes too long when compared to training.

MarcioCoelho
MarcioCoelho Dataiku DSS Core Designer, Registered Posts: 12 ✭✭✭✭

Hey everyone,

I've recently ran into a small issue, which is the time it takes to dataiku to score a model.

As you can see in provided screenshot, it took the model roughly 9min to do everything from loading to saving, while the remainder 51min were spent in scoring it.

Is there anything I can do to speed this up? I imagine that, for a binary classification, it calculates the results using different cut-off thresholds, but I'm not sure.

The algorithm I am using is a custom python model, based on a standard scikit-learn algorithm which is BalancedRandomForest Classifier.

Any help would be appreciated.

Best regards,

Márcio


Operating system used: Windows

Answers

  • mkamal1
    mkamal1 Registered Posts: 1 ✭✭✭✭

    @MarcioCoelho
    I have the same problem after upgrading DSS 9.0 to 10.0.4 on Linux. It happens across all visual ML models (Random Forests, XGBoost, LightGBM, etc).

    The model is training a binary classifier. Training takes about 5 minutes, but the scoring is stuck for over an hour. I am using DSS's builtin environment for visual ML LightGBM.

    Did you find a fix?

  • MarcioCoelho
    MarcioCoelho Dataiku DSS Core Designer, Registered Posts: 12 ✭✭✭✭

    Unfortunately no, the issue still persists, even after upgrading to DSS 10.0.4 (I had the issue in version 9.0).

Setup Info
    Tags
      Help me…