IME - Building an Emotion Classification System on Videos
Name: Mohamed AbdElAziz Khamis Omar
Title: Senior Data Scientist
Description: We are providing data management solutions including Data integration, Data Quality, Master Data Management, Data Archiving, Data Masking, Big Data, Test Data Management, Data Governance, Operational Data Store, Data Warehouse and Business Intelligence. IME has customers in Saudi Arabia, United Arab Emirates, Qatar, Kuwait, Oman, Egypt, Iraq, Algeria, Tunisia, Morocco, Nigeria, Pakistan, Bangladesh, and the Maldives. For more information about IME please visit us at infme.com
Our data science challenges revolve around:
Maintaining the code
Building end-to-end data science projects
Using state-of-the-art pre-trained models
This project is the developer version of the "Dev Day - Dataiku + phData Workbook".
We built an emotion classification system on videos using the Dataiku deep learning for images plugin, which allows us to download pre-trained deep learning networks and provides recipes such as image classification retraining and scoring to classify emotions for the videos. This plugin uses Tensorflow and Keras on Python for image classification.
The approach that we will take for video classification is to break each emotion video into a fixed number of frames and then use these images to train a deep residual neural network (known as resnet) to classify emotions within each image.
This resnet network has been previously trained on the ImageNet dataset, so we do not have to start it from scratch.
Finally, we’ll evaluate the predicted emotion for a video through taking a majority vote on labels predicted across all its frames.
Image Classification RecipeProject WorkflowPrepare the Emotion Classification DatasetFrames sampled from each video in Emotion imagesFilename identifiers
Train the Deep Learning Model
Score Frames and Evaluate Labels for Emotion Videos
Build the Production Pipeline
Avg. of prediction_calm by frame1Emotion Classification Engine
Code maintainability: Dataiku provides a high degree of code maintainability through visual recipes and visual workflows.
Ease-of-use: The platform made it very easy to build the end-to-end data science project.
Upskilling: We were able to leverage state-of-the-art pre-trained models through a wide range of macros and plugins.