Deep Learning Training Error

Forermx
Forermx Dataiku DSS Core Designer, Registered Posts: 2
edited July 16 in Using Dataiku

I'm getting the following error when I try to begin training the deep learning model

Traceback (most recent call last):
  File "/app/dataiku_design/dataiku-dss-9.0.3/python/dataiku/doctor/server.py", line 46, in serve
    ret = api_command(arg)
  File "/app/dataiku_design/dataiku-dss-9.0.3/python/dataiku/doctor/dkuapi.py", line 45, in aux
    return api(**kwargs)
  File "/app/dataiku_design/dataiku-dss-9.0.3/python/dataiku/doctor/commands.py", line 424, in train_prediction_keras
    pipeline.generated_features_mapping)
  File "/app/dataiku_design/dataiku-dss-9.0.3/python/dataiku/doctor/prediction_entrypoints.py", line 462, in prediction_train_model_keras
    generated_features_mapping)
  File "/app/dataiku_design/dataiku-dss-9.0.3/python/dataiku/doctor/deep_learning/keras_support.py", line 117, in get_keras_model
    base_callbacks)
  File "<string>", line 48, in fit_model
  File "/app/dataiku_design/dssdata-9.0.3/code-envs/python/tissue_segmentation/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/app/dataiku_design/dssdata-9.0.3/code-envs/python/tissue_segmentation/lib/python3.6/site-packages/keras/engine/training.py", line 1732, in fit_generator
    initial_epoch=initial_epoch)
  File "/app/dataiku_design/dssdata-9.0.3/code-envs/python/tissue_segmentation/lib/python3.6/site-packages/keras/engine/training_generator.py", line 185, in fit_generator
    generator_output = next(output_generator)
  File "/app/dataiku_design/dssdata-9.0.3/code-envs/python/tissue_segmentation/lib/python3.6/site-packages/keras/utils/data_utils.py", line 625, in get
    six.reraise(*sys.exc_info())
  File "/app/dataiku_design/dssdata-9.0.3/code-envs/python/tissue_segmentation/lib/python3.6/site-packages/six.py", line 719, in reraise
    raise value
  File "/app/dataiku_design/dssdata-9.0.3/code-envs/python/tissue_segmentation/lib/python3.6/site-packages/keras/utils/data_utils.py", line 610, in get
    inputs = future.get(timeout=30)
  File "/utilities/python/3.6.13/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
  File "/utilities/python/3.6.13/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/app/dataiku_design/dssdata-9.0.3/code-envs/python/tissue_segmentation/lib/python3.6/site-packages/keras/utils/data_utils.py", line 406, in get_index
    return _SHARED_SEQUENCES[uid][i]
  File "/app/dataiku_design/dataiku-dss-9.0.3/python/dataiku/doctor/deep_learning/sequences.py", line 186, in __getitem__
    return self.process_batch_func(self.original_sequence[index])
  File "/app/dataiku_design/dataiku-dss-9.0.3/python/dataiku/doctor/deep_learning/sequences.py", line 219, in _process_batch_func
    new_X_batch[inp] = DataAugmentationSequence.duplicate_rows(X_batch, n_augmentation)
  File "/app/dataiku_design/dataiku-dss-9.0.3/python/dataiku/doctor/deep_learning/sequences.py", line 192, in duplicate_rows
    return np.tile(array, tuple([n_augmentation] + [1] * (len(array.shape) - 1)))
AttributeError: 'dict' object has no attribute 'shape'

 

I can't find an instance where I may have set up something wrong so I'm not sure what is wrong


Operating system used: Windows 10 Pro

Answers

  • Sarina
    Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 317 Dataiker

    Hi @Forermx
    ,

    Thank you for reporting this! This is unfortunately a known issue when using a DataAugmentationSequence for deep learning when you also have a non-augmented feature in your training. The main option for now would be to remove the DataAugmentationSequence or the non-augmented feature from the training. This has been reported to our engineering team. Please let me know if you have any questions in the meantime.

    Thank you,
    Sarina

Setup Info
    Tags
      Help me…