IME - Building an Emotion Classification System on Videos
mohamed-khamis
Partner, L2 Admin, L2 Designer, Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Snowflake Advanced, Dataiku DSS Adv Designer, Registered, Dataiku DSS Developer, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant Posts: 13 Partner
Name:
Mohamed AbdElAziz Khamis Omar
Title:
Senior Data Scientist
Country:
Egypt
Organization:
IME
Description:
We are providing data management solutions including Data integration, Data Quality, Master Data Management, Data Archiving, Data Masking, Big Data, Test Data Management, Data Governance, Operational Data Store, Data Warehouse and Business Intelligence.
IME has customers in Saudi Arabia, United Arab Emirates, Qatar, Kuwait, Oman, Egypt, Iraq, Algeria, Tunisia, Morocco, Nigeria, Pakistan, Bangladesh, and the Maldives. For more information about IME please visit us at infme.com
Awards Categories:
- Alan Tuning
Challenge:
Our data science challenges revolve around:
- Maintaining the code
- Building end-to-end data science projects
- Using state-of-the-art pre-trained models
Solution:
This project is the developer version of the "Dev Day - Dataiku + phData Workbook".
We built an emotion classification system on videos using the Dataiku deep learning for images plugin, which allows us to download pre-trained deep learning networks and provides recipes such as image classification retraining and scoring to classify emotions for the videos. This plugin uses Tensorflow and Keras on Python for image classification.
The emotion classification dataset is the speech video data from RAVDESS, and an example of one of the videos can be found on YouTube.
The approach that we will take for video classification is to break each emotion video into a fixed number of frames and then use these images to train a deep residual neural network (known as resnet) to classify emotions within each image.
This resnet network has been previously trained on the ImageNet dataset, so we do not have to start it from scratch.
Finally, we’ll evaluate the predicted emotion for a video through taking a majority vote on labels predicted across all its frames.
Impact:
- Code maintainability: Dataiku provides a high degree of code maintainability through visual recipes and visual workflows.
- Ease-of-use: The platform made it very easy to build the end-to-end data science project.
- Upskilling: We were able to leverage state-of-the-art pre-trained models through a wide range of macros and plugins.