Pilot Evaluation Report: Evaluation of the Durham Works Futures programme

Summary

We funded Durham County Council (DCC) as a part of our Development Stream of What Works funded programmes.

DCC delivered Durham Works Futures (DWF), which is part of the local authority-led, multi-agency partnership that supports young people at risk of NEET in transition from education, and those already NEET into a sustained employment outcome. The specific focus is young people with experience of Children’s Social Care, SEN Support, exclusion from education, or  Alternative Provision.

We were particularly interested in the inclusion of Intermediate Labour Market (ILM) support as a part of this intervention, where participants were provided access to temporary, part-time, or full-time waged jobs, subsidised by DCC, with ongoing support from employment coaches throughout. This report highlights that this was seen as a very positive first step toward sustained employment by employers, employment coaches, and participants.

Study aims:

• Identify how the support model operates and the programme’s Theory of Change
• Support DCC to understand which elements of delivery work most effectively and why​
• Support improvements to the programme and data collection processes, to build organisational capacity and assess the feasibility of further evaluation.​
• Assess the costs and benefits of the programme

Evaluation approach

The evaluation included a process study to develop and test the Theory of Change, understand participant outcomes, and identify any refinements which could be made to improve delivery.

The mixed-method evaluation drew on evidence from:

• monitoring and outcomes data
• a repeated survey for participants
• a cost analysis survey for employment coaches
• qualitative interviews with 16 young people (3 at multiple timepoints), 6 frontline staff, 3 strategic staff, 2 employers
• multi-media digital diaries from 7 participants
• observations and staff workshops

The study also aimed to understand the feasibility of conducting an impact evaluation of the Durham Works Futures programme.

Key insights

The study found that there were associations between higher levels of engagement with Durham Works Futures and a higher likelihood of transition into EET outcomes, and improvements in wellbeing and self-esteem, compared with a comparator group drawn from Understanding Society. The qualitative work suggests that Durham Works Futures helps participants to achieve a range of positive outcomes such as improved confidence and self-esteem, social and interpersonal skills, and increased knowledge of the job market and job searching skills.

The study highlights that a main driver of these outcomes is the mentor-mentee relationship, which is a key element of the intervention. Specifically, the study highlights that the holistic nature of support, and focused work on increasing participants’ confidence to engage in opportunities was a driver of outcomes.

Aside from these outcome findings, the study found that the programme is delivered as intended and as indicated by the Theory of Change. Although the participant journeys could vary considerably from person to person, the phases of the intervention align with the Participant Journey map.

The Cost-Benefit Analysis study found that there are positive net benefits associated with Durham Works Futures. The cost-benefit ratio was 1.6, meaning that the total benefit associated with the programme was 1.6 times greater than the total cost.

Next steps  

We commissioned a feasibility study alongside this work to investigate the potential for an experimental or quasi-experimental design study. The feasibility report highlighted several challenges regarding the suitability of progressing this programme to an impact study. A randomised control trial was found to not be suitable to the delivery partners.

In consideration of a quasi-experimental design, the predominant challenge was difficulty in accessing a suitable comparison group within available administrative datasets or within alternative local authorities, and no option providing high enough quality evidence to outweigh the risks.