Names:
Lottie Norton, Impact and Analysis Manager, with:
Meirin Evans
Julie Cummings
Lauren Bellaera
Sam Lewis
Annabel Marcuse
Country: United Kingdom
Organization: The Brilliant Club
The Brilliant Club is a UK-wide charity that mobilises the PhD community to support less advantaged students to access the most competitive universities and succeed when they get there. We deliver The Scholars Programme in schools across the UK. Our PhD tutors share their subject knowledge and passion for learning with small groups of students aged 8-18. Students experience university-style learning and complete a challenging final assignment to build the knowledge, skills and confidence needed to progress to university. Independent analysis demonstrates programme graduates are significantly more likely to progress to competitive universities than their peers (56% vs 37%).
Awards Categories:
- Data Science for Good
- Responsible AI
- Most Impactful Transformation Story
- Most Impactful Ikigai Story
- Excellence in Research
Business Challenge:
The Brilliant Club is a UK-wide charity that mobilises the PhD community to support less advantaged students to access the most competitive universities and succeed when they get there.
We exist because students from the least advantaged backgrounds have a 3 in 100 chance of going to the most competitive universities, compared with 33 in 100 for the most advantaged (UCAS Multiple Equality Measure). This disadvantage doesn’t disappear when they enter university. Disadvantaged students are also 3x more likely to miss out on a 1st or 2:1 degree.
Graduates from the UK’s most competitive universities are more likely to access professional careers (The Sutton Trust, 2019) and earn £10,000 more annually than their peers (Department for Education, 2019). We want to create a world in which every student has a fair chance to access the life-changing opportunities that come with attending a competitive university.
Our Scholars Programme sees PhD researchers share their subject knowledge and passion for learning with state school students aged 8-18, so they can develop the knowledge, skills, and confidence to secure a place at a competitive university.
Students complete a university-style final assignment, which is often the first time they have completed work of this length, or used skills such as referencing or independent research. Scholars Programme graduates are invited to celebrate their achievements at a graduation ceremony at one of our partner universities and take part in information, advice, and guidance sessions with undergraduates. Independent UCAS analysis demonstrates that students who complete a final assignment are more likely to progress to a competitive university than their peers who did not participate on the programme (56% vs 37%). However, the problem we face is that each year a minority of students do not submit their final assignment, meaning they miss out on important aspects of the programme.
Business Solution:
This project was a collaborative effort, involving roles across the organisation. The programme management team, who are the on-the ground running of The Scholars Programme, initially raised the need to understand why some students don’t submit their final assignment and the importance of supporting them to do so. One data scientist from the Research and Impact Department led on the build and implementation of the project over a 3-month period, while another member was on hand to offer consultation and support. They worked together with a volunteer data scientist from Dataiku (via the Ikig.ai programme) to explore how we could use the Dataiku Online platform to generate actionable insights.
Having explored the data cleaning features of Dataiku and the functionality to produce descriptive statistics, we identified the platform’s predictive modelling capabilities as having the potential to help us answer our question and increase our impact. Working with the programme management team, we built a predictive model that outputs a ‘risk rating’ for each student. We did this by building gradient-boosted decision trees (81% accuracy) that produced a high, medium, or low risk score, indicating a student’s likelihood of not submitting their final assignment.
Dataiku offered the unique opportunity to not only allow us to build a complex model using python coding, but also set it up in such a way that non-specialist colleagues could re-run the model using the visual flow and the ability to easily upload data. Had we not had access to the platform we would not have been able to continue re-running the model in the same way, as it would have taken too many hours of staff time when data science capacity within the organisation is limited and needed on other projects. The transparency of the flow and the replicability of the model have been vital to ensuring the sustainability of the project and we ran the ‘risk rating’ model at six time points across the 2021/22 academic year.
The ease of understanding the outputs has been fundamental to the project’s success. We have communicated the risk ratings back to the programme management team, who have in turn created guidance for the tutors and schoolteachers we work with on how to support the students most at risk of not submitting.
Business area enhanced: Communication/Strategy/Competitive Intelligence
Use case stage: Built & Functional
Impact:
This project has strengthened our approach to how we use data to inform decision-making and improve service delivery. For example, the outcomes of this project have been shared with the Senior Leadership Team and we have integrated use of Dataiku’s model into our UK-wide programme work that reaches more than 14,000 students per year. The findings have also been used to create guidance and training for more than 400 PhD tutors delivering The Scholars Programme.
The proportion of students submitting their final assignment has increased since we integrated ‘risk scores’ into our programme delivery cycle. Almost 8000 students have completed the challenging university-style assignment, gaining important transferable skills and knowledge through the process. As more students have completed the programme, a greater number have had the opportunity to attend a graduation trip at one of the UK’s most competitive universities. We will continue to use the model to shape our programme delivery and as part of this will monitor the impact on final assignment submissions.
We know that students who complete their final assignment progress to a competitive university at a higher rate than their peers who do not take part in the programme (56% vs 37%). As such, we would expect that the increased final assignment submission rate will lead to more less advantaged students progressing to a competitive university, meaning they experience the life changing opportunities associated with higher education. We await further data to confirm this.
Specific Value Brought by Dataiku:
The seamless integration of code and point-and-click analysis has increased team efficiency and the speed at which we can produce meaningful findings from complex analysis. Thanks to the replicability of the model, we will be able able to run it in future academic years, meaning the project will have a sustained impact. Meanwhile, Dataiku’s features mean we can maintain the model and continuously improve it in an agile way, pinpointing specific areas for improvement.
This project has enabled us to highlight the impact of collaborating with specialist organisations to generate data insights – see this published blog. This is a particularly valuable message for the university access sector because evaluation expertise is variable and can be under-resourced. Therefore, this project has helped to strengthen the message externally about the need for data and evaluation.
The expert advice and support from Dataiku’s volunteer data scientist and access to a myriad of training resources have been instrumental in upskilling our Research and Impact Department and programme management team in machine learning and data analytics. Building on our work with Dataiku, we have started to run internal training sessions on data analysis and research methods. The aim of the series is to increase non-technical colleagues’ data literacy in different parts of the organisation.
Value range: Thousands of $
