Detailresult - Fully Automating the Production of Daily Predictions for Over 100 Food Retail Stores
Name:
Martijn Schuit
Chantal van Dok
Teun IJntema
Sirik Scherer
Tané van der Zwet
Country: Netherlands
Organization: Detailresult
Detailresult is the service organization providing IT, finance, and HR services to the supermarket formulas of Dirk van den Broek and Dekamarkt. Both of these formulas have been active in the Netherlands for over 80 years. They are both family-owned businesses with different approaches to food retail but a shared sense of entrepreneurship. Detailresult aims to support these businesses in their day-to-day operations and help them deliver the best value to their customers.
Awards Categories:
- Best MLOps Use Case
Business Challenge:
In the food retail industry, there is always a trade-off between having enough stock to maximize revenue and having the minimal amount of stock to satisfy customer demand. Not having enough stock leads to lost sales, while having too much stock results in waste due to products reaching their expiration date. This is an especially challenging problem in the fresh bread department. Freshly baked bread, as the name suggests, is freshly baked each day in the store and is not allowed to be sold the day after it was baked. This means that if you have more stock than you sell on a given day, then you would have to throw away the unsold bread.
Prediction models can be very useful in predicting the fluctuating customer demand and help us to have just the right amount of bread in stock. We have found that using a SARIMAX model gave us the best predictions for customer demand. The problem is that the best predictions were the results of a model trained for a specific store and bread type combination. Having more than 100 stores and several different bread types, this means that we would need to have hundreds of models in production to produce daily predictions for each store and bread type combination.
As you can imagine, having hundreds of models in production running daily would be very challenging to maintain. Dataiku helped us solve this problem.
Business Solution:
Training and maintaining hundreds of models can be very challenging. Dataiku helped us to fully automate the data ingestion pipeline, model training scripts, model evaluation scripts, and production of daily predictions. The project had been in development for more than a year before we started moving to the Dataiku platform. We had multiple technical pilots and commercial pilots in the stores. All the model training and prediction scripts had to be kicked off manually during those pilots. We quickly realized that it would not be feasible to have this in production while also moving on to new projects. This would take up all of our time.
The Dataiku platform allowed us to fully automate the data pipelines and model flows. If a new store is opened, it is automatically detected, and a new model is specifically trained for it. New sales data is automatically loaded and used for predictions for the following day.
Day-to-day Change:
Previously, the bakers from each store had to manually predict demand. With various factors changing customer demand, it was nearly impossible to make a good prediction by hand. Now, we supply predictions to the stores, which they can alter if they expect something else based on their personal experiences or other factors not used in the models. This results in stores having to spend less time ordering bread and more time focusing on products that require a little more attention and expertise to predict correctly, such as items that are on promotion that week.
Aside from the improvements in the store, Dataiku also massively improved our team's ability to monitor performance using the dashboard possibilities within Dataiku as well as the automated emailing regarding errors and warnings. This changed our response from being reactive to proactive and helped us handle problems before they became an issue in the stores.
Business Area: Supply Chain/Supplier Management/Service Delivery
Use Case Stage: In Production
Value Generated:
The metrics that were used to determine the success of the project were:
- The number of hours that products were out of stock (OOS).
- The amount of waste.
- The number of corrections that were made to the predictions by the stores as an indicator of the amount of time spent on the ordering process.
The project resulted in an almost 30% reduction in OOS hours, a 10% reduction in waste, and an average of 10-15 minutes of time saved each day in over 100 stores. With the amount of bread sold daily, the waste reduction is a big help in reaching the goals of the food retail industry to reduce their food waste significantly by 2030.
Value Brought by Dataiku:
The value brought by Dataiku lies mostly in the time saved on maintaining the project. Because we have an automation and a design environment, it is very easy to create a fix for a bug or a problem in the design environment and deploy that in the automation environment using the automation bundles that Dataiku provides. The data pipelines were also easily used or replicated for other projects which required the same data or logic, which increased the effectiveness of the entire team and helped us to develop new projects at a much faster rate than before. Additionally, the monitoring possibilities in the form of dashboards and automated warnings give the team the ability to anticipate problems and handle them proactively.
Value Type:
- Reduce cost
- Save time
Value Range: Hundreds of thousands of $
Comments
-
Hi,
Can you elaborate on what functionalities of Dataiku you used to develop the model?
How did you train the 100+ models every day?
