Error in Deep learning in images plugin

Swapnali
Swapnali Registered Posts: 38 ✭✭✭✭
in Plugins & Extending Dataiku

Hi Team.

I am following below link to achieve emotion analysis - https://www.phdata.io/dataiku-deep-learning-tutorial-emotion-classification-in-videos/

I performed 10 epochs with 10 steps in every epochs while using 'Retraining image classification model'

I am getting following error while using 'Image classification'. kindly help us to solve this error

eaerr.PNG

· Share on FacebookShare on Twitter

Answers

  • arnaudde
    arnaudde Dataiker Posts: 52 Dataiker

    Hello,

    It looks like the images your are trying to score with your model are not of the right format.

    What images do you use as input for the image classification recipe ?

    Could you share one for further investigation ?

    Arnaud

    · Share on FacebookShare on Twitter
  • Swapnali
    Swapnali Registered Posts: 38 ✭✭✭✭

    Thanks Arnaud for your quick reply,

    For my experiment I am using the dataset mentioned in that link.

    and following each step explained in that link.

    · Share on FacebookShare on Twitter
  • Swapnali
    Swapnali Registered Posts: 38 ✭✭✭✭

    Hi Arnaud,

    I am waiting for your reply. Kindly provide you help

    · Share on FacebookShare on Twitter
  • arnaudde
    arnaudde Dataiker Posts: 52 Dataiker

    Hello Swapnali,
    Something has probably gone wrong while doing the tutorial. Can you share the input image that you feed to the image classification recipe ?

    Without more information on the input of the plugin I am afraid I can't help.

    Best

    · Share on FacebookShare on Twitter
  • Swapnali
    Swapnali Registered Posts: 38 ✭✭✭✭

    Following is the flow we have created

    Swapnali_0-1610010470831.png

    In this Emotion videos is the Input folder in which we have multiple mp4 input files.

    Swapnali_1-1610010470840.png

    The link we used to download the input files is - https://zenodo.org/record/1188976/files/Video_Song_Actor_01.zip?download=1

    · Share on FacebookShare on Twitter
  • arnaudde
    arnaudde Dataiker Posts: 52 Dataiker

    Hello,
    Thanks for the additional information.

    Could you show what is in your "Testing images files" folder ? Can you share one of the images inside it ?

    · Share on FacebookShare on Twitter
  • Swapnali
    Swapnali Registered Posts: 38 ✭✭✭✭

    Swapnali_0-1610012214551.png

    This is what we have in Testing Images files

    following is the image

    01-02-03-02-01-01-01_happy_f65.png

    · Share on FacebookShare on Twitter
  • arnaudde
    arnaudde Dataiker Posts: 52 Dataiker

    Thanks for the information
    The problem is probably that your images are not at the root of the folder but in the sub directory "/Actor"

    Could you try the Image classification recipe again with a folder with images at the root ?

    · Share on FacebookShare on Twitter
  • Swapnali
    Swapnali Registered Posts: 38 ✭✭✭✭

    Thanks Arnaud,

    It worked!!

    However now I am preparing ScoredImages with following python code

    def process(row):
    max_val = 0
    max_label = None

    emotions = ['calm', 'sad', 'surprised', 'neutral', 'fearful', 'angry', 'happy', 'disgust']

    for e in emotions:
    p = float(row['prediction_{}'.format(e)])
    if p > max_val:
    max_val = p
    max_label = e

    row['max_prediction'] = max_val
    row['max_label'] = max_label
    row['label'] = row['images'].split('_')[1]
    row['correct'] = 1 if row['label'] == row['max_label'] else 0
    row['video_path'] = '/{}.mp4'.format(row['images'].split('_')[0])

    return row

    and I am getting following error. kindly help to solve

    eaerr4.PNG

    · Share on FacebookShare on Twitter
  • arnaudde
    arnaudde Dataiker Posts: 52 Dataiker

    Hello,

    You don't have a column named prediction_calm, probably because your network never predicted an image as "calm".

    Best

    · Share on FacebookShare on Twitter
  • Swapnali
    Swapnali Registered Posts: 38 ✭✭✭✭

    some images are labeled as Calm. you can see

    eaerr5.PNG

    eaerr6.PNG

    · Share on FacebookShare on Twitter
  • arnaudde
    arnaudde Dataiker Posts: 52 Dataiker

    The error you are encountering is because there is no "prediction_calm" column in your dataset.

    What are the columns of the dataset you are applying this step on ?

    · Share on FacebookShare on Twitter
  • Swapnali
    Swapnali Registered Posts: 38 ✭✭✭✭

    There are Images, Prediction and error columns are in ScoredImages dataset which is input dataset.

    and according to the steps in link , ScoredImagesPrepared (Output dataset)should have a new column max_label

    eaerr5.PNG

    · Share on FacebookShare on Twitter
  • arnaudde
    arnaudde Dataiker Posts: 52 Dataiker

    You need to use the unnest processor on the prediction column first.

    It will create all the prediction_ columns and therefore the custom python processor will be able to find the "prediction_calm" key of row.

    You can learn more about python processors in the doc

    · Share on FacebookShare on Twitter
  • robertcoop
    robertcoop Partner, Dataiku DSS Core Designer, Registered Posts: 2 Partner

    Thanks for helping out, arnaudde!

    · Share on FacebookShare on Twitter
  • Swapnali
    Swapnali Registered Posts: 38 ✭✭✭✭

    Hi Arnaud,

    We tried to understand the document you sent .But not able to understand .

    Could you help me out with an appropriate processor for our use case?

    Thanks a lot in advance

    · Share on FacebookShare on Twitter
  • arnaudde
    arnaudde Dataiker Posts: 52 Dataiker
    edited July 17

    Hello Swapnali,


    I think this code will achieve what you want.

    Replace the code suggested in the phdata tutorial at step 3.C: Extract Predicted Labels for Images with the following one:

    import json
def process(row):
         max_val = 0
         max_label = None

         emotions = ['calm', 'sad', 'surprised', 'neutral',  'fearful', 'angry', 'happy', 'disgust']

         for e in emotions:
                p = float(json.loads(row['prediction'])['prediction_{}'.format(e)])
                if p > max_val:
                      max_val = p
                      max_label = e

        row['max_prediction'] = max_val
        row['max_label'] = max_label
        row['label'] = row['images'].split('_')[1]
        row['correct'] = 1 if row['label'] == row['max_label'] else 0
        row['video_path'] = '/{}.mp4'.format(row['images'].split('_')[0])

        return row

    · Share on FacebookShare on Twitter
  • Swapnali
    Swapnali Registered Posts: 38 ✭✭✭✭

    Hi Arnaud,

    I am still getting same kind of error. I have attached error in text file

    · Share on FacebookShare on Twitter
  • arnaudde
    arnaudde Dataiker Posts: 52 Dataiker
    edited July 17

    My bad the key is "calm" not "prediction_calm" below code should work

    import json
def process(row):
         max_val = 0
         max_label = None

         emotions = ['calm', 'sad', 'surprised', 'neutral',  'fearful', 'angry', 'happy', 'disgust']

         for e in emotions:
                p = float(json.loads(row['prediction'])[e])
                if p > max_val:
                      max_val = p
                      max_label = e

        row['max_prediction'] = max_val
        row['max_label'] = max_label
        row['label'] = row['images'].split('_')[1]
        row['correct'] = 1 if row['label'] == row['max_label'] else 0
        row['video_path'] = '/{}.mp4'.format(row['images'].split('_')[0])

        return row
    · Share on FacebookShare on Twitter
  • Swapnali
    Swapnali Registered Posts: 38 ✭✭✭✭

    thanks Arnaud,

    Now we are facing error like code is taking keys Only from first row it seems. In first row there is no surprised, angry and disgust keys available so it is throwing errors like: same for angry and disgust

    dataikuerr.PNG

    row is like:

    dataikuerr1.PNG

    · Share on FacebookShare on Twitter
  • arnaudde
    arnaudde Dataiker Posts: 52 Dataiker
    edited July 17

    The processor is iterating on each row but your predictor does not output probas for every emotion on every line.
    I added a if statement to check if the emotion is in the prediction

    import json
def process(row):
         max_val = 0
         max_label = None

         emotions = ['calm', 'sad', 'surprised', 'neutral',  'fearful', 'angry', 'happy', 'disgust']

         for e in emotions:
            if e in json.loads(row['prediction']):
                p = float(json.loads(row['prediction'])[e])
                if p > max_val:
                      max_val = p
                      max_label = e

        row['max_prediction'] = max_val
        row['max_label'] = max_label
        row['label'] = row['images'].split('_')[1]
        row['correct'] = 1 if row['label'] == row['max_label'] else 0
        row['video_path'] = '/{}.mp4'.format(row['images'].split('_')[0])

        return row
    · Share on FacebookShare on Twitter
  • robertcoop
    robertcoop Partner, Dataiku DSS Core Designer, Registered Posts: 2 Partner

    Alternatively, you can adjust the scoring recipe to provide all 8 classification labels as outputs.

    · Share on FacebookShare on Twitter
  • Swapnali
    Swapnali Registered Posts: 38 ✭✭✭✭

    Thanks a lot Arnaud for your extended help. I completed the entire pipeline

    · Share on FacebookShare on Twitter
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…