Getting people out of debt with Christians Against Poverty
“I’m really super excited about where we can go from here. If we do want to help more people, there’s absolutely no question that data can be a big part of that.”
Martin Cowles
Senior Project Manager, Christians Against Poverty
Christians Against Poverty (CAP) helps people escape unmanageable debt. Their mission is ‘to release thousands of families from grinding poverty through debt counselling and community groups’. This is usually done through face-to-face meetings with debt coaches who are based in their network of churches and a team of trained Debt Advisors who help their clients plan how to manage their debts.
CAP’s debt support programme is extremely effective, helping 2,337 people become debt-free in 2020 alone. But Debt Advisors need a lot of personal information from new clients before they can recommend an appropriate route out of debt. This is both labour-intensive for the counsellor and can feel overwhelming for some clients, who may already be in a vulnerable state.
Despite the high level of support Advisors provide, with regular reminders and check-ins, many clients get ‘stuck’ and are slow to make progress, or simply leave the process. Because of this, CAP is beginning a transformation of its debt operations – specifically reconsidering how they engage with clients at early stages. The CAP team wanted to use their existing client data to improve their processes by better managing staff workloads, predicting the best support routes for their clients, and identifying at-risk people more quickly, to ultimately support more people out of debt.
What are CAP’s data challenges?
The CAP charity representatives and DataKind UK volunteer Data Ambassadors broke their challenge down into three ‘data-scienceable’ goals for DataDive weekend volunteers:
Understand which pieces of information about a client are key for predicting the debt reduction route they will be recommended to make the early stages of each client’s journey easier for them and their counsellor.
Determine which support strategies are most effective in helping clients along their journey, to help more people escape debt more quickly.
Predict the level of support needed based on client profiles, in order to target high-risk cases.
CAP also hoped that the results of this project would highlight the value of data-driven decisions to their whole organisation.
Martin Cowles, Senior Project Manager at CAP, commented, “From a cultural point of view, I think the project could be a massive springboard in terms of the appetite in the organisation for data-driven decision making and the application of data science. Both the director of debt operations and our chief executive have already expressed really strong interest in investing in this area going forward – which is a huge win.”
Main findings
About half of CAP’s Debt Advisors’ assumptions about their clients, based on the information provided at the beginning of their journeys, were true.
But certain beliefs about client groups, such as whether pensioners are more likely to take insolvency, were shown to be completely false.
Accurately predicting the recommended route for a client is possible using only a few pieces of intake information, so CAP can choose to collect less information from clients.
There are some tasks that CAP’s clients struggle with more than others.
There are some strong indicators of which cases will be high-risk (meaning the client is slow to progress and the case is resource intensive). These include the reason for debt, the region, the length of the repayment terms, and the amount being paid in monthly.
Understanding what information is key to advising clients accurately
When someone first interacts with CAP, they have to provide a lot of information, such as their total debt or number of children. Advisors then recommend a best route for each client, which might be a payment plan, insolvency, or a ‘holding route’ if their circumstances are still changing. The information gathering process can be stressful for clients and create bottlenecks for staff, and CAP wondered whether all of it was actually helpful for determining a client’s route.
To start with, DataDive volunteers explored some assumptions CAP had about which route – payment, insolvency, or holding – would be recommended to different clients. For example, CAP suspected that clients over pensionable age (66) were more likely to be recommended insolvency. But analysis of their data showed that the opposite was true, and more people under 66 (51%) were advised to take insolvency than pensioners (28%).
Knowing that staff assumptions weren’t always true, the volunteers next investigated which pieces of ‘intake’ information are really critical to deciding the best route for a client. Two approaches they used were logistic regression models and decision tree classification. Regression analysis is a way of statistically determining which factors have the biggest impact on the outcome (here, the route taken) and which factors can be ignored. Decision tree classification models are similar to flowcharts – here the model learns to classify clients into different routes based on answers to a series of questions, e.g. “Is the client a pensioner?”
Excitingly, both techniques were able to accurately predict a client’s route up to 90% of the time with only a few pieces of information. This graph shows how the amount of information affected the accuracy of the predictions for one of the models. With only three pieces of information it correctly predicted a client’s route nearly 75% of the time; with eight pieces this accuracy rose to 80%. After that point more information didn’t improve the accuracy much further, shown by the flattening curve. The specific intake information that was most helpful varied depending on which approach was used.
The CAP team can now decide which pieces of information are easiest for clients to provide in order for their Advisors to still give accurate advice, which will help to make the intake process easier, faster, and less stressful for the client.
Jonny Sisterson, HOPE Technician at CAP, said that the findings from this DataDive will “provide really valuable insights … ensuring that we are changing things based on the data that we have, and not on the assumptions that we have”
Predicting the level of support needed by different types of clients
CAP’s support requires a lot of resources and staff time, with high workloads that limit the amount of people they can help. To address this, the team aimed to determine what factors were associated with a higher-than-expected workload. For the analysis, CAP considered workload on a 0-4 point scale, 0 units being the least amount of work and 4 being the most. For each case, the volunteers estimated the expected workload and actual workload – the unexpected workload is the discrepancy between these two numbers. They then tried to determine which factors were associated with a higher or lower unexpected workload.
A client’s housing status and reason for debt both strongly influenced the unexpected workload, but surprisingly, so did their region. In the figure below, the bars show the average amount of additional workload in each region, and the black line indicates the range of expected additional work. For example, while the average unexpected workload was about 1 unit in the West Midlands, it ranged between approximately 0-3 units. Cases from Northern Ireland were associated with less work than expected, but those from the South of England were associated with about 7 additional units!
Identifying these factors could help staff to balance the types of clients they support, and budget their time more efficiently. In the future, CAP could build on this principle to prioritise cases, reduce counsellor workloads, and get faster outcomes for their clients.
As the weekend ended, Martin reminded us all that, “Sometimes when we’re going through the data, you can forget that every line on there is a human who might be facing an absolute world of darkness now, caused not just by debt, but loneliness, isolation, or health problems. Your input this weekend is a step to helping us to do our jobs better… A massive thank you on behalf of our clients.”
What’s next
Jake Hutton, Director of Debt Operations at CAP, said, “Our DataDive weekend was a stunning weekend for us organisationally. The team that came in to help were geniuses and with no background knowledge they hit the ground running and helped us uncover some fantastic new findings!”
The DataDive helped CAP’s team understand how to plan and execute their own analysis in future, what data is most useful, and what tools they might use. They gained some clearer ideas about data collection principles that will influence their coming debt service transformation project, and provide quick wins with policies and priorities for their clients. CAP now has a very exciting set of findings that should help them to support more people to successfully leave debt behind. The team is keen to share the findings in order to get the whole organisation more invested in data, and to hire a full-time data scientist.
Martin added, “I’m really super excited about where we can go from here. CAP’s got a journey ahead of us in the next few years, and we want to help more people. And if we do want to help more people, there’s absolutely no question that data can be such a big part of that.”
With huge thanks to the DataDive team: volunteer Data Ambassadors (and absolute superstars) Alicia Mergenthaler, Jakub Orwat-Kapola, Cal Mcauliffe, Claire Bénard and troubleshooter Luisa Pirés worked with CAP staff members Martin, Jonny, Rebecca, Dave, and Helen.
