Shiju Nair, Sr. Engineer, Digital IA Technologies, Audit and Assurance
Ahmed Abujarad, SVP, Audit and Assurance
Darsan Krishnan, Manager, Group Quality Assurance and Excellence
Muzamil Riffat, Manager, Digital & Technology Audit
Niladri Das, Head, Group Specialized Audit Division
Habib Ahmed Noor, Senior Specialist, Procurement, Governance and Excellence
Mohammed K Al Mansoori, Manager Group Digital Commercial & Financial Solutions
Antonio Rivas , Senior Specialist Group Digital Advanced Analytics Solutions
Country: United Arab Emirates
Organization: Abu Dhabi National Oil Company (ADNOC)
Established in 1971, Abu Dhabi National Oil Company (ADNOC) is a diversified group of energy and petrochemical companies that employs more than 50,000 people and is a major contributor to the GDP of the United Arab Emirates (UAE). ADNOC operate across the entire hydrocarbon value chain with different fully integrated Group Companies providing services in the fields of exploration and production; oil refining and gas processing; chemicals and petrochemicals; refined products and distribution; maritime transportation; and support services including sales and marketing, human capital, legal, finance and IT. ADNOC has been named the UAE’s most valuable brand for a second consecutive year starting from 2019, 28.6% increase over the previous year and 145% increase since the launch of its transformation strategy in 2017, making it the fastest-growing brand in the Middle East and the first UAE brand to surpass $10 billion in value.
ADNOC Audit and Assurance Function(s) is responsible for providing a wide domain of Auditing services across ADNOC Head Quarters and 16+ Group Companies.
The current situation and key challenges faced by the audit team include the following:
ADNOC has successfully implemented a unified SAP system (“ONE ERP”) across all entities. Group Audit Excellence team within Audit & Assurance Function embarked upon an initiative to implement Continuous Control Assurance (CCA) to capitalize on the system’s unified data repository. The team set out with the following key goals:
The CCA journey started in 2021 with conceptualizing the model and obtaining the necessary alignment from business counterparts. An end-to-end architectural design was established from data ingestion to visualization prior to the development.
Dataiku enabled the team to perform all stages of the project cycle under a centralized platform as it has a wide range of capabilities which includes preparation of data pipelines, visualization, machine learning and advanced analytics, governance, and collaboration.
Data Acquisition – Data Source connections were setup across the source system especially SAP ERP and the Supplier database. Around 65 data tables were acquired for data pre-processing.
Data Standardization and Enrichment – With Dataiku’s Prepare recipe, data fields were standardized to a common format to help prepare complex joining of datasets for further transformation.
Data Transformation and Processing – Visual recipes supported with data transformations like data joining, grouping, custom field generation provided the implementation team significant ability during design phase. A few recipes used in the flow are Data Preparation, SQL recipe, Python Recipe, and Machine Learning Recipes. The team managed over 350+ datasets and number of visual recipes for transformation, rolling aggregations and various data handling processes shared across projects.
Automated Workflow Scenarios – Dataiku scenarios are a great feature in automating and scheduling data pipelines managed dependencies. Workflows are scheduled at daily and monthly frequencies along with email notifications supported to alert users upon failures, which helps IT in taking proactive maintenance measures.
Machine Learning Model for Test Classification– Machine learning models were trained and deployed to predict and classify test results under data quality, process and policy were developed using Dataiku Auto ML mode. Dataiku’s rich visual aids and features like hyperparameter, metric selection and variable dependence and chart-based analytics supported in retraining and saving of models to improve performance and help select the best algorithm.
Implementation of Continuous Control Assurance has given management greater visibility into timely information on business processes and enabled increased coordination with Internal Audit and IT. Noteworthy benefits of CCA solution powered by Dataiku includes:
Dataiku’s role has been pivotal in building and delivering the CCA solution, some of the key value proposition is highlighted below:
Reduction in Delivery Time - Easy to build workflows using a visual interface with ready-made recipes of Dataiku enabled the building of data processes much faster. Automation of repeatable data pipelines supported in saving time and reduced any rework upon data changes.
Easy to use Interface for Complex Data Processing - Data integration tasks like moving data, integrating data from multiple source systems, managing complex logic supported by data quality processes was much easier and even novice team were able to build data driven projects.
Process Transparency and Data Lineage - Supported by visual recipes, AI preview and extensive data processing logs, the data processing is quite transparent, and team can get an end-to-end view of data flow along with relevant statistic.
Centralized Shared Platform - Being a web-based development platform it was easy to share project with other team members for reuse. Over and above version control feature of Dataiku enabled integrity of the project code developed by different teams intact.
Quick Turn-Around in Development of Machine Learning Models - GUI rich resources and components available in Dataiku lab extensively supported in the design and development of Machine learning models using multiple candidate algorithms with a detailed statistical analysis on the dependent variables.