Community Conundrum 25:Feature Visualization is now live! Read More

No model trained by this step (train_model())

Level 2
No model trained by this step (train_model())

I am using function train_model() in the Scenario as following syntax, however, the error"No model trained by this step" shows up in the log. 

train_ret = scenario.train_model("0pwbYbmy")
trained_model = train_ret.get_trained_model()
performance = train_ret.get_new_version_metrics().get_performance_values()
if performance["RMSE"] < best_rmse:
trained_model.activate_new_version()

 

Traceback (most recent call last):  File "/home/dataiku/dss/scenarios/NY_TAXI_LJ_1/RETRAIN_ML_MORE_SAMPLING/2020-06-14-11-44-25-612/custom-scenario/script.py", line 37, in <module>    trained_model = train_ret.get_trained_model()  File "/home/dataiku/dataiku-dss-7.0.2/python/dataiku/scenario/step.py", line 197, in get_trained_model    raise ValueError("No model trained by this step")ValueError: No model trained by this step

 Manually train the model from interface can success. any idea why?

 

Thanks,

Lj

0 Kudos
8 Replies
Dataiker
Dataiker

Hi Lj,

Could you tell if you can see if you see your model in the run logs? 

There should be something like this:

 

(Line starting with "Built model...")

 

Screenshot 2020-06-15 at 22.28.04.png

 

Also do you see a new version of a model appear in a Saved model 0pwbYbmy  ?

 

Regards,

Andrey

Andrey Avtomonov
R&D Engineer @ Dataiku
Level 2
Author

Hi Andrey,

I've checked the log, there is a line looks like below. But there is no new version appeared in my Saved model. 

The .log file has been attached, could you help check on it?

Start step build model 0pwbYbmy

 Capture.PNG

Thanks,

Lj

0 Kudos
Dataiker
Dataiker

Hi @zhlj ,

 

Could you also try to create a new "Custom python script" based scenario with only this code:

from dataiku.scenario import Scenario
scenario = Scenario()
train_ret = scenario.train_model("0pwbYbmy")

 

and check if it leads to the new model version appearing inside the saved model.

 

Also when you said that training worked manually, did you run the retrain recipe from the flow?

 

Regards,

Andrey

Andrey Avtomonov
R&D Engineer @ Dataiku
0 Kudos
Level 2
Author

Andrey, 

with the three line of your code, the scenario can run successfully, as shown in the picture. and yes, manaully means I retrain model from retrain recipe in the flow

Capture.PNG

0 Kudos
Level 2
Author

and when I try to add two more lines of code, that error shows up again:

 

from dataiku.scenario import Scenario

scenario = Scenario()
train_ret = scenario.train_model("0pwbYbmy")

trained_model = train_ret.get_trained_model()
performance = trained_model.get_new_version_metrics().get_performance_values()

 

0 Kudos
Level 2
Author

it worked... andrey, but I changed nothing?  at the very beginning there is a typo error like below:

trained_model = train_ret.get_trained_model()
performance = train_ret.get_new_version_metrics().get_performance_values()

But even I change "train_ret" back to "trained_model" and run the script, I still got that error.....  

Now i run the script again, it surprisingly worked... I got confused ..

Anyway, I have one more question, with the syntax "trained_model = train_ret.get_trained_model()", the trained model get activated automatically even though it's performance factor is not the highest, can I stop that, so I can compare its performance factor, if it is better than previous model, than activate it. 

 

Thanks,

Lj

0 Kudos
Dataiker
Dataiker

Hi Lj,

Regarding the active version initialization, it's actually this line that sets an active version after training:

train_ret = scenario.train_model("0pwbYbmy")

This is a default behaviour, every new version is activated automatically. You can change it by setting "Activate new versions" parameter in a saved model settings to "Manually"

And then activate the best model with your python code.

Regards,

Andrey

Andrey Avtomonov
R&D Engineer @ Dataiku
Level 2
Author

Thanks, this works.