Error in Deep learning in images plugin
Hi Team.
I am following below link to achieve emotion analysis - https://www.phdata.io/dataiku-deep-learning-tutorial-emotion-classification-in-videos/
I performed 10 epochs with 10 steps in every epochs while using 'Retraining image classification model'
I am getting following error while using 'Image classification'. kindly help us to solve this error
Answers
-
Hello,
It looks like the images your are trying to score with your model are not of the right format.
What images do you use as input for the image classification recipe ?
Could you share one for further investigation ?
Arnaud
-
Thanks Arnaud for your quick reply,
For my experiment I am using the dataset mentioned in that link.
and following each step explained in that link.
-
Hi Arnaud,
I am waiting for your reply. Kindly provide you help
-
Hello Swapnali,
Something has probably gone wrong while doing the tutorial. Can you share the input image that you feed to the image classification recipe ?Without more information on the input of the plugin I am afraid I can't help.
Best
-
Following is the flow we have created
In this Emotion videos is the Input folder in which we have multiple mp4 input files.
The link we used to download the input files is - https://zenodo.org/record/1188976/files/Video_Song_Actor_01.zip?download=1
-
Hello,
Thanks for the additional information.Could you show what is in your "Testing images files" folder ? Can you share one of the images inside it ?
-
This is what we have in Testing Images files
following is the image
-
Thanks for the information
The problem is probably that your images are not at the root of the folder but in the sub directory "/Actor"Could you try the Image classification recipe again with a folder with images at the root ?
-
Thanks Arnaud,
It worked!!
However now I am preparing ScoredImages with following python code
def process(row):
max_val = 0
max_label = None
emotions = ['calm', 'sad', 'surprised', 'neutral', 'fearful', 'angry', 'happy', 'disgust']
for e in emotions:
p = float(row['prediction_{}'.format(e)])
if p > max_val:
max_val = p
max_label = e
row['max_prediction'] = max_val
row['max_label'] = max_label
row['label'] = row['images'].split('_')[1]
row['correct'] = 1 if row['label'] == row['max_label'] else 0
row['video_path'] = '/{}.mp4'.format(row['images'].split('_')[0])
return rowand I am getting following error. kindly help to solve
-
Hello,
You don't have a column named prediction_calm, probably because your network never predicted an image as "calm".
Best
-
some images are labeled as Calm. you can see
-
The error you are encountering is because there is no "prediction_calm" column in your dataset.
What are the columns of the dataset you are applying this step on ?
-
There are Images, Prediction and error columns are in ScoredImages dataset which is input dataset.
and according to the steps in link , ScoredImagesPrepared (Output dataset)should have a new column max_label
-
You need to use the unnest processor on the prediction column first.
It will create all the prediction_ columns and therefore the custom python processor will be able to find the "prediction_calm" key of row.
You can learn more about python processors in the doc
-
Thanks for helping out, arnaudde!
-
Hi Arnaud,
We tried to understand the document you sent .But not able to understand .
Could you help me out with an appropriate processor for our use case?
Thanks a lot in advance
-
Hello Swapnali,
I think this code will achieve what you want.Replace the code suggested in the phdata tutorial at step 3.C: Extract Predicted Labels for Images with the following one:
import json def process(row): max_val = 0 max_label = None emotions = ['calm', 'sad', 'surprised', 'neutral', 'fearful', 'angry', 'happy', 'disgust'] for e in emotions: p = float(json.loads(row['prediction'])['prediction_{}'.format(e)]) if p > max_val: max_val = p max_label = e row['max_prediction'] = max_val row['max_label'] = max_label row['label'] = row['images'].split('_')[1] row['correct'] = 1 if row['label'] == row['max_label'] else 0 row['video_path'] = '/{}.mp4'.format(row['images'].split('_')[0]) return row -
Hi Arnaud,
I am still getting same kind of error. I have attached error in text file
-
My bad the key is "calm" not "prediction_calm" below code should work
import json def process(row): max_val = 0 max_label = None emotions = ['calm', 'sad', 'surprised', 'neutral', 'fearful', 'angry', 'happy', 'disgust'] for e in emotions: p = float(json.loads(row['prediction'])[e]) if p > max_val: max_val = p max_label = e row['max_prediction'] = max_val row['max_label'] = max_label row['label'] = row['images'].split('_')[1] row['correct'] = 1 if row['label'] == row['max_label'] else 0 row['video_path'] = '/{}.mp4'.format(row['images'].split('_')[0]) return row -
thanks Arnaud,
Now we are facing error like code is taking keys Only from first row it seems. In first row there is no surprised, angry and disgust keys available so it is throwing errors like: same for angry and disgust
row is like:
-
The processor is iterating on each row but your predictor does not output probas for every emotion on every line.
I added a if statement to check if the emotion is in the predictionimport json def process(row): max_val = 0 max_label = None emotions = ['calm', 'sad', 'surprised', 'neutral', 'fearful', 'angry', 'happy', 'disgust'] for e in emotions: if e in json.loads(row['prediction']): p = float(json.loads(row['prediction'])[e]) if p > max_val: max_val = p max_label = e row['max_prediction'] = max_val row['max_label'] = max_label row['label'] = row['images'].split('_')[1] row['correct'] = 1 if row['label'] == row['max_label'] else 0 row['video_path'] = '/{}.mp4'.format(row['images'].split('_')[0]) return row -
Alternatively, you can adjust the scoring recipe to provide all 8 classification labels as outputs.
-
Thanks a lot Arnaud for your extended help. I completed the entire pipeline
