ModuleNotFoundError: No module named 'modellightgbm'

laurent_temo

I'm training a LightGBM classification model on Dataiku using a Python 3.6 environment with the following packages on dataiku 13.3.1 version :

scikit-learn>=0.20,<0.21
scipy>=1.2,<1.3
xgboost==1.5.2
statsmodels>=0.10,<0.11
jinja2>=2.10,<2.11
flask>=1.0,<1.1
cloudpickle>=1.3,<1.6
matplotlib==3.3.4
numpy==1.19.5
GPyOpt==1.2.6
plotly==5.14.1
nbformat>=4.2.0
colorthief==0.2.1
joblib==1.1.1
lightgbm==3.3.5
pandarallel==1.6.4
ipywidgets==7.7.5
tqdm==4.64.1
Pillow==8.4.0
parquet==1.3.1
pyarrow==6.0.1
pyjanitor==0.23.1
The training process works fine because my training environment includes a custom module named modellightgbm (probably a wrapper around LightGBM). However, when I use the default prediction recipe (predict model) to generate scores, I receive the following error :

[dataiku.base.dku_pickle] Couldn't unpickle modellightgbm.dku_lightgbm.DkuLGBMClassifier, retrying with modellightgbm.dku_lightgbm.DkuLGBMClassifier
ModuleNotFoundError: No module named 'modellightgbm'

This appears to indicate that the prediction environment cannot find the modellightgbm module, which prevents the trained model from being unpickled. So I want to add this module to the project code space (for example in the lib or python folder) so that it is accessible during prediction.

Alexandru

Hi,

Have you tried adding modellightgbm?
Python 3.6 is quite old and will be deprecated in the future.
Can you try upgrading your code environments to at least python3.9 and retrain?
You can change the verison directly from the code env e.g Change Python interpreter make sure you package are compatible with Pyton3.9 you can use recommended presets for VisualML