ModuleNotFoundError: No module named 'modellightgbm'
I'm training a LightGBM classification model on Dataiku using a Python 3.6 environment with the following packages on dataiku 13.3.1 version :
scikit-learn>=0.20,<0.21
scipy>=1.2,<1.3
xgboost==1.5.2
statsmodels>=0.10,<0.11
jinja2>=2.10,<2.11
flask>=1.0,<1.1
cloudpickle>=1.3,<1.6
matplotlib==3.3.4
numpy==1.19.5
GPyOpt==1.2.6
plotly==5.14.1
nbformat>=4.2.0
colorthief==0.2.1
joblib==1.1.1
lightgbm==3.3.5
pandarallel==1.6.4
ipywidgets==7.7.5
tqdm==4.64.1
Pillow==8.4.0
parquet==1.3.1
pyarrow==6.0.1
pyjanitor==0.23.1
The training process works fine because my training environment includes a custom module named modellightgbm (probably a wrapper around LightGBM). However, when I use the default prediction recipe (predict model) to generate scores, I receive the following error :
[dataiku.base.dku_pickle] Couldn't unpickle modellightgbm.dku_lightgbm.DkuLGBMClassifier, retrying with modellightgbm.dku_lightgbm.DkuLGBMClassifier
ModuleNotFoundError: No module named 'modellightgbm'
This appears to indicate that the prediction environment cannot find the modellightgbm module, which prevents the trained model from being unpickled. So I want to add this module to the project code space (for example in the lib or python folder) so that it is accessible during prediction.