Error in Deep learning in images plugin

Swapnali
Level 3
Error in Deep learning in images plugin

Hi Team.

I am following below link to achieve emotion analysis - https://www.phdata.io/dataiku-deep-learning-tutorial-emotion-classification-in-videos/

I performed 10 epochs with 10 steps in every epochs while using 'Retraining image classification model'

I am getting following error while using 'Image classification'. kindly help us to solve this error

eaerr.PNG 

 

0 Kudos
23 Replies
arnaudde
Dataiker

Hello,

It looks like the images your are trying to score with your model are not of the right format.

What images do you use as input for the image classification recipe ?

Could you share one for further investigation ?

Arnaud

0 Kudos
Swapnali
Level 3
Author

Thanks Arnaud for your quick reply,

For my experiment I am using  the dataset mentioned in that link.

and following each step explained in that link.

 

0 Kudos
Swapnali
Level 3
Author

Hi Arnaud,

I am waiting for your reply. Kindly provide you help

0 Kudos
arnaudde
Dataiker

Hello Swapnali,
Something has probably gone wrong while doing the tutorial. Can you share the input image that you feed to the image classification recipe ?

Without more information on the input of the plugin I am afraid I can't help.

Best

0 Kudos
Swapnali
Level 3
Author

Following is the flow we have created

Swapnali_0-1610010470831.png

 

In this Emotion videos is the Input folder in which we have multiple mp4 input files. 

Swapnali_1-1610010470840.png

The link we used to download the input files is - https://zenodo.org/record/1188976/files/Video_Song_Actor_01.zip?download=1

0 Kudos
arnaudde
Dataiker

Hello,
Thanks for the additional information. 

Could you show what is in your "Testing images files" folder ? Can you share one of the images inside it ?

0 Kudos
Swapnali
Level 3
Author

 

Swapnali_0-1610012214551.png

 

This is what we have in Testing Images files

following is the image 

01-02-03-02-01-01-01_happy_f65.png

0 Kudos
arnaudde
Dataiker

Thanks for the information
The problem is probably that your images are not at the root of the folder but in the sub directory "/Actor"

Could you try the Image classification recipe again with a folder with images at the root ?

0 Kudos
Swapnali
Level 3
Author

Thanks Arnaud,

It worked!!

However now I am preparing ScoredImages with following python code 

def process(row):
         max_val = 0
         max_label = None

         emotions = ['calm', 'sad', 'surprised', 'neutral',  'fearful', 'angry', 'happy', 'disgust']

         for e in emotions:
                p = float(row['prediction_{}'.format(e)])
                if p > max_val:
                      max_val = p
                      max_label = e

        row['max_prediction'] = max_val
        row['max_label'] = max_label
        row['label'] = row['images'].split('_')[1]
        row['correct'] = 1 if row['label'] == row['max_label'] else 0
        row['video_path'] = '/{}.mp4'.format(row['images'].split('_')[0])

        return row

and I am getting following error. kindly help to solve

eaerr4.PNG

 

 
0 Kudos
arnaudde
Dataiker

Hello,

You don't have a column named prediction_calm, probably because your network never predicted an image as "calm".

Best

0 Kudos
Swapnali
Level 3
Author

some images are labeled as Calm. you can see

eaerr5.PNG

eaerr6.PNG

0 Kudos
arnaudde
Dataiker

The error you are encountering is because there is no "prediction_calm" column in your dataset. 

What are the columns of the dataset you are applying this step on ?

0 Kudos
Swapnali
Level 3
Author

There are Images, Prediction and error columns are in ScoredImages dataset which is input dataset. 

and according to the steps in link , ScoredImagesPrepared (Output dataset)should  have a new column max_label

eaerr5.PNG

0 Kudos
arnaudde
Dataiker

You need to use the unnest processor on the prediction column first.

It will create all the prediction_ columns and therefore the custom python processor will be able to find the "prediction_calm" key of row.

You can learn more about python processors in the doc 

0 Kudos
robertcoop
Level 1

Thanks for helping out, arnaudde!

0 Kudos
Swapnali
Level 3
Author

Hi Arnaud, 

We tried to understand the document you sent .But not able to understand .

Could you help me out with an appropriate  processor for our use case?

Thanks a lot in advance

0 Kudos
arnaudde
Dataiker

Hello Swapnali,


I think this code will achieve what you want.

Replace the code suggested in the phdata tutorial at step 3.C: Extract Predicted Labels for Images with the following one:

import json
def process(row):
         max_val = 0
         max_label = None

         emotions = ['calm', 'sad', 'surprised', 'neutral',  'fearful', 'angry', 'happy', 'disgust']

         for e in emotions:
                p = float(json.loads(row['prediction'])['prediction_{}'.format(e)])
                if p > max_val:
                      max_val = p
                      max_label = e

        row['max_prediction'] = max_val
        row['max_label'] = max_label
        row['label'] = row['images'].split('_')[1]
        row['correct'] = 1 if row['label'] == row['max_label'] else 0
        row['video_path'] = '/{}.mp4'.format(row['images'].split('_')[0])

        return row

 

0 Kudos
Swapnali
Level 3
Author

Hi Arnaud,

I am still getting same kind of error. I have attached error in text file

0 Kudos
arnaudde
Dataiker

My bad the key is "calm" not "prediction_calm" below code should work

import json
def process(row):
         max_val = 0
         max_label = None

         emotions = ['calm', 'sad', 'surprised', 'neutral',  'fearful', 'angry', 'happy', 'disgust']

         for e in emotions:
                p = float(json.loads(row['prediction'])[e])
                if p > max_val:
                      max_val = p
                      max_label = e

        row['max_prediction'] = max_val
        row['max_label'] = max_label
        row['label'] = row['images'].split('_')[1]
        row['correct'] = 1 if row['label'] == row['max_label'] else 0
        row['video_path'] = '/{}.mp4'.format(row['images'].split('_')[0])

        return row
0 Kudos
Swapnali
Level 3
Author

thanks Arnaud,

Now we are facing error like code is taking keys Only from first row it seems. In first row there is no surprised, angry and disgust keys available so it is throwing errors like: same for angry and disgust

dataikuerr.PNG

row is like:

dataikuerr1.PNG

 

0 Kudos