Using Keras in Dataiku throws error

Highlighted
malexs
Level 1
Using Keras in Dataiku throws error

I'm using the dataiku VM and have installed Keras and Tensorflow in the dataiku python environment.



I can create a code notebook in the Lab and use the code and data from the following guide and it works http://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/



However if I try to use the exact same code within a "Custom Python Model" under Analysis, I always get the error 



Training Failed… Sorry

global name 'Sequential' is not defined



 



The stacktrace in run/backend.log is:





[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - 2017-07-17 20:07:19,939 INFO Fitting model:
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - 2017-07-17 20:07:19,939 INFO Model is: <keras.wrappers.scikit_learn.KerasClassifier object at 0x7f4c4cca53d0>
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - 2017-07-17 20:07:19,939 INFO train_X class: <type 'numpy.ndarray'>
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - 2017-07-17 20:07:19,939 INFO train_X shape: (123, 4)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - 2017-07-17 20:07:19,939 INFO train_y shape: (123,)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - 2017-07-17 20:07:19,939 INFO END - Fitting model
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - 2017-07-17 20:07:19,939 ERROR API call failed : None
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - Traceback (most recent call last):
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/server.py", line 87, in do_POST
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - ret = api_command(arg)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/dkuapi.py", line 45, in aux
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - return api(**kwargs)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/commands.py", line 253, in train_prediction_models_nosave
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - pipeline)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/prediction_entrypoints.py", line 43, in prediction_train_score_save
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - (clf, out_params, prepared_X, iipd) = classification_fit(modeling_params, split_desc, transformed_train, prediction_type)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/prediction/classification_fit.py", line 584, in classification_fit
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - clf.fit(train_X, train_y, **kwargs)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - File "/home/dataiku/dss/condaenv/lib/python2.7/site-packages/keras/wrappers/scikit_learn.py", line 201, in fit
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - return super(KerasClassifier, self).fit(x, y, **kwargs)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - File "/home/dataiku/dss/condaenv/lib/python2.7/site-packages/keras/wrappers/scikit_learn.py", line 138, in fit
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - self.model = self.build_fn(**self.filter_sk_params(self.build_fn))
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - File "<string>", line 16, in create_model
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - NameError: global name 'Sequential' is not defined
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - 2017-07-17 20:07:19,939 WARNING Handle failed
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils] - 127.0.0.1 - - [17/Jul/2017 20:07:19] "POST /command HTTP/1.1" 500 -
[2017/07/17-20:07:19.940] [Exec-304] [INFO] [dku.utils] - ('SCIKIT MODEL', <keras.wrappers.scikit_learn.KerasClassifier object at 0x7f4c4cca53d0>)
[2017/07/17-20:07:19.940] [MRT-339] [ERROR] [dku.analysis.prediction] - Processing failed
com.dataiku.common.server.APIError$APIErrorException: global name 'Sequential' is not defined
at com.dataiku.common.rpc.InternalAPIClient.handleStringResp(InternalAPIClient.java:629)
at com.dataiku.common.rpc.InternalAPIClient.postFormToString(InternalAPIClient.java:435)
at com.dataiku.dip.analysis.coreservices.AnalysisMLKernel.executeCommand(AnalysisMLKernel.java:103)
at com.dataiku.dip.analysis.ml.prediction.PyRegularNoSavePredictionHandler$TrainAdditionalThread.process(PyRegularNoSavePredictionHandler.java:118)
at com.dataiku.dip.analysis.ml.shared.PRNSTrainThread.run(PRNSTrainThread.java:66)


 



 

2 Replies
UserBird Dataiker
Dataiker
Re: Using Keras in Dataiku throws error
It looks like you're missing some imports - in this case Sequential
0 Kudos
malexs
Level 1
Re: Using Keras in Dataiku throws error
Unfortunately not. It has the exact same code between the notebook and the custom module. ie. they both include "from keras.models import Sequential"

It can run the code "model = Sequential()" directly in the custom python module without issues, so I think its something to do with the fact that the baseline_module() method which calls Sequential() is passed as a parameter in the line "estimator = KerasClassifier(build_fn=baseline_model, ..." and then attempted to be executed in a different space...
0 Kudos
Labels (2)