Names:
Stéphane Raguideau, GEM IS Digital & Data Accelerator
Yasine Meridja, GEM IS Digital & Data Accelerator
Country: France
Organization: ENGIE - Global Energy Management & Sales
Global Energy Management & Sales (GEMS) is one of ENGIE’s Global Entity. At the heart of the energy value chain, we optimize the Group’s assets portfolio including electricity, renewable technologies, natural gas, environmental products and bulk commodities such as biomass. We also develop our own external commercial franchise worldwide and rely on four main expertise to offer tailor-made, innovative and competitive solutions. We provide services in energy supply & global commodities, energy transition services, risk management & market access, and asset management. With a staff of 1,400, offices in 15 countries including 8 main spots, GEMS has an extended geographical coverage in Europe, the US and Asia-Pacific.
Awards Categories:
- Value at Scale
- Most Impactful Transformation Story
Business Challenge:
Data science is at the core of our activities at ENGIE - Global Energy Management & Sales. Users across departments manage various sources of data, including:
- Energy consumption,
- Market data,
- Weather information,
- Deal and order book,
- Etc.
This data is leveraged by the business for many purposes, including:
- Pricing,
- Risk management,
- Data reconciliation from various sources,
- Reporting,
- Etc.
But access to the data was limited due to its sheer volume, security considerations, and tooling segmentation. In addition, coding skills were required for accessing it, which excluded many users who did not have a technical background.
Users needed to manually retrieve the data through a variety of applications, which caused several issues:
- Task repetitiveness, which was very time-consuming - and namely included extracting data from the different systems in place.
- Data availability, as all data sources were not always referenced and only IT may have been able access these.
- Operational risk, which is related to the quality of the data and the manual processing taking place (e.g. mistakes in copy/pasting steps),
- Coding skills required to manipulate the data and automatize part of the process, e.g. “Visual Basic for Applications” (VBA) in Excel, or Python.
- Tooling was not fit for the volume of data (in particular, Excel).
Business Solution:
We implemented Dataiku in 2018. As with every new tool, Dataiku requires specific onboarding to maximize its benefits. At GEMS, our users have different profiles and backgrounds, hence they are not all familiar with data manipulation and analysis.
It is therefore important to provide them with training opportunities, regardless of their division (trading, risk, back office, finance, IT, etc.). This includes:
- Understanding their needs and identifying a use case to conduct a Proof of Concept (POC).
- Developing the most relevant training in regard to their profile and skills.
- Building the Dataiku plugins and connectors to allow them to easily and securely access the data.
- Hosting regular workshops (at least once per week) on select topics throughout the POC, including partitioning, Python recipes, machine learning, automation, dashboarding, pattern recognition, etc.)
This training path is set to two months, after which users are given autonomy to access the data, manipulate it for their day-to-day needs, and most importantly, are able to explore new areas to gain more insights from their data, which is a key pillar of data democratization within GEMS.
We then empower these users across business teams and monitor their (and our) success through two key elements:
- Community (Yammer + dedicated Dataiku instance)
- Dataiku Watcher, an internal tool enabling:
- Dataiku Instance Monitoring
- Adoption Report
- License Management
Community (Yammer + Dataiku Instance)
An important key to success is to transmit the desire to share knowledge, to collaborate and to facilitate user learning. To make this possible, we started a Dataiku Community with different communication channels:
Yammer, where we post articles around new Dataiku version, internal trainings, Quiz about Dataiku usage, Dataiku white paper and even the movie Data Science Pioneers: Conquering the Next Frontier. Every post has a relaxed tone and humoristic GIF and encourages readers to add comments.
A dedicated community Dataiku instance where are stored Internal trainings, plugins how-to's, and Dataiku project to pass Dataiku certification exam. All projects contain various descriptions, very detailed Wiki with many screenshots and comments and are structured with Flow Zone and Tags.
Dataiku Watcher
Becoming a data scientist or a data engineer doesn’t happen overnight though, hence we’ve developed a tool to manage our Dataiku instances, monitor all projects created, and ensure they’re following established governance and best practices - including data connections, scenarios, data sharing, partitioning, plugin types, etc.) All users are therefore able to produce insights safely!
Dataiku Watcher has been designed to easily manage all our Dataiku Instances and monitor:
- Dataiku Instance monitoring
- Adoption report
- Licenses management (costs reporting by entity/team and license costs efficiency)
Dataiku Instance Monitoring
For each of our Dataiku Instance installed on our servers, we are able to monitor:
- Connections over the last month
- Adoption metrics & License usage
- Number of project designed by our end users
- Etc.
Each project is analyzed (datasets, recipes, metadata, connections, plugins usages, automation, interaction with other projects & team, etc.) and a score is computed over 3 different lines: Collaboration/Design/Automation.
The score reflects the fact that the project matches the internal guidance and best practices promoted internally.
Thanks to this score, we can detect potential problems in advance and work with the teams by providing them with tailor-made training.
Adoption Report
How Dataiku is used across the organization? How many hours do our users spend on the platform and how many times do they log in per month?
All this questions, and more, are essentials to ensure that Dataiku is still used, fit our needs, and is still relevant in our data tools framework. Dataiku Watcher provide all the required metrics on a daily basis.
License Management
Dataiku Watcher provides tools to ensure every license purchased is used. Inactive Data Scientist are detected (no activity over the last 60 days) allowing us to save on licensing costs by reallocating unused licenses (if any) instead of purchasing new ones.
Value Brought by Dataiku:
Dataiku enables us to drive positive impact through the following features:
- It has never been so easy to access & consume data:
- through a number of plugins created internally, which enable users to easily and securely interact with the different data sources
- through CDH (Common Data Hub, ENGIE Group Data platform based on AWS S3): thousands of datasets produced by all the entities of the Group are accessible in 1 click
- Low code/no code data manipulation: visual recipes enable users to prepare and transform the data to fit their needs, without any coding skills required.
- For more complex operations, the collaborative visual interface enables our IT teams to work hand-in-hand with the business on building and editing workflows.
- Sharing insights from the data is made easy with the dashboarding features.
- Process automation, leading to:
- Shortened time-to-market, now that reporting and analysis are available on-demand.
- Increased monitoring capabilities, as monthly and weekly analysis can easily be turned into daily reports.
- Reduced operational risk, as manual operations are now automatized.
Impact:
Dataiku enables business teams to scale their impact and save time across a broad range of operations:
“We save 1-2 hours a day when producing our liquidity risk indicators and we now have the possibility to focus our efforts on enriching our liquidity analysis instead”
Treasury and Liquidity Risk Officer
“We've automatized and accelerated our month-end closing process from 5 days to 15 minutes”
FinOps Project Manager
“The number of mispriced derivatives has decreased by approximately 66%, which saves us about 3 hours per day”
Lead Weather Derivatives Structurer
“Dataiku reduces the time for back testing our models from 1 week to 10 minutes. Additionally, the platform helps us understand each step of the analysis”
Weather Derivatives Structurer
