Using Keras in Dataiku throws error

malexs
malexs Registered Posts: 7 ✭✭✭✭
edited July 16 in Using Dataiku

I'm using the dataiku VM and have installed Keras and Tensorflow in the dataiku python environment.

I can create a code notebook in the Lab and use the code and data from the following guide and it works http://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/

However if I try to use the exact same code within a "Custom Python Model" under Analysis, I always get the error

Training Failed… Sorry

global name 'Sequential' is not defined

The stacktrace in run/backend.log is:



[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 INFO Fitting model:
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 INFO   Model is: <keras.wrappers.scikit_learn.KerasClassifier object at 0x7f4c4cca53d0>
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 INFO   train_X class: <type 'numpy.ndarray'>
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 INFO   train_X shape: (123, 4)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 INFO   train_y shape: (123,)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 INFO END -  Fitting model
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 ERROR API call failed : None
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - Traceback (most recent call last):
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/server.py", line 87, in do_POST
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     ret = api_command(arg)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/dkuapi.py", line 45, in aux
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     return api(**kwargs)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/commands.py", line 253, in train_prediction_models_nosave
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     pipeline)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/prediction_entrypoints.py", line 43, in prediction_train_score_save
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     (clf, out_params, prepared_X, iipd) = classification_fit(modeling_params, split_desc, transformed_train, prediction_type)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dataiku-dss-4.0.5/python/dataiku/doctor/prediction/classification_fit.py", line 584, in classification_fit
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     clf.fit(train_X, train_y, **kwargs)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dss/condaenv/lib/python2.7/site-packages/keras/wrappers/scikit_learn.py", line 201, in fit
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     return super(KerasClassifier, self).fit(x, y, **kwargs)
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "/home/dataiku/dss/condaenv/lib/python2.7/site-packages/keras/wrappers/scikit_learn.py", line 138, in fit
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -     self.model = self.build_fn(**self.filter_sk_params(self.build_fn))
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  -   File "<string>", line 16, in create_model
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - NameError: global name 'Sequential' is not defined
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 2017-07-17 20:07:19,939 WARNING Handle failed
[2017/07/17-20:07:19.940] [Exec-305] [INFO] [dku.utils]  - 127.0.0.1 - - [17/Jul/2017 20:07:19] "POST /command HTTP/1.1" 500 -
[2017/07/17-20:07:19.940] [Exec-304] [INFO] [dku.utils]  - ('SCIKIT MODEL', <keras.wrappers.scikit_learn.KerasClassifier object at 0x7f4c4cca53d0>)
[2017/07/17-20:07:19.940] [MRT-339] [ERROR] [dku.analysis.prediction]  - Processing failed
com.dataiku.common.server.APIError$APIErrorException: global name 'Sequential' is not defined
        at com.dataiku.common.rpc.InternalAPIClient.handleStringResp(InternalAPIClient.java:629)
        at com.dataiku.common.rpc.InternalAPIClient.postFormToString(InternalAPIClient.java:435)
        at com.dataiku.dip.analysis.coreservices.AnalysisMLKernel.executeCommand(AnalysisMLKernel.java:103)
        at com.dataiku.dip.analysis.ml.prediction.PyRegularNoSavePredictionHandler$TrainAdditionalThread.process(PyRegularNoSavePredictionHandler.java:118)
        at com.dataiku.dip.analysis.ml.shared.PRNSTrainThread.run(PRNSTrainThread.java:66)

Tagged:
· Share on FacebookShare on Twitter

Answers

  • UserBird
    UserBird Dataiker, Alpha Tester Posts: 535 Dataiker
    It looks like you're missing some imports - in this case Sequential
    · Share on FacebookShare on Twitter
  • malexs
    malexs Registered Posts: 7 ✭✭✭✭
    Unfortunately not. It has the exact same code between the notebook and the custom module. ie. they both include "from keras.models import Sequential"

    It can run the code "model = Sequential()" directly in the custom python module without issues, so I think its something to do with the fact that the baseline_module() method which calls Sequential() is passed as a parameter in the line "estimator = KerasClassifier(build_fn=baseline_model, ..." and then attempted to be executed in a different space...
    · Share on FacebookShare on Twitter
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…