A co-produced evaluation of Scope’s Support to Work service

Support to Work is an online and telephone support programme for disabled people who are applying for jobs.

Jessica Bricknell, Amy Frounks, Ruth Murran and Giles Skerry

We would like to express our thanks to several people and organisations who helped make this co-produced evaluation possible.

Skip to Acknowledgements

This report presents an evaluation of Support to Work using data that describes customer and staff experiences from January to December 2019. But, we analysed data and wrote this report as Covid-19 spread across the UK.

The coronavirus pandemic has changed what employment looks like for many people. It has also affected how people access services. Scope has adapted many of its services to the new environment. It remains to be seen how permanent these changes will be, but we would like to briefly mention how Support to Work has been affected by coronavirus so far. We also consider the potential effects of coronavirus for disabled people looking for employment.

Support to Work and coronavirus

As a service normally provided through digital and telephone channels, Support to Work was perfectly positioned to continue providing its service into lockdown with minimal disruption. The Support to Work team started and continue to work from home.

During the first weeks of lockdown, fewer people than normal referred themselves to the service, though this drop in referrals was not as large as the ones seen in Scope’s face-to-face employment services. The number of referrals has since risen again, with the team receiving a record number of referrals in June 2020.

The service’s marketing has been updated to emphasise its remote delivery methods. The Support to Work team have also worked with Scope’s Information and Advice team to create new guidance on video interviews, which a lot of people looking for work will now experience as part of the application process.

During appointments, advisers have particularly supported customers with interview skills and preparation for starting work remotely. Support to Work customers are still successfully entering work during this atypical time.

The team have received feedback that the service has been a welcome source of structure to a home routine that has changed a lot. For those with caring commitments, it has been a time to focus on oneself and feel a sense of achievement.

Disabled people, employment, and coronavirus

We also wish to acknowledge the complex potential effect of coronavirus on disabled people and employment.

In one sense, coronavirus has increased workplace accessibility for many. Early in the pandemic, employers responded swiftly to set up or expand remote and flexible working practices. Now this precedent has been set, we would argue that it will be much more difficult for employers to deny flexible working and working from home as reasonable adjustments for disabled people. This could have a particularly positive effect on disabled people in remote areas, or in areas where accessible job opportunities are low.

The pandemic has also highlighted the importance of access to digital technology and skills to keep us connected and maintain wellbeing. We discuss in this report that digital access and skills can be a barrier for disabled people looking for work. We hope the pandemic has substantially increased employers’ and government’s awareness of and commitment to bridging the digital divide.

At the same time, as lockdown relaxes and financial support for employers and those who are shielding is withdrawn, people of working age who are shielding may come under increasing pressure to return to work. Many will face incredibly difficult decisions between their health and employment. The predicted recession is likely to have a significant effect on disabled people’s prospects of staying in work.

Along with the reintroduction of unemployment benefit sanctions and disability benefit reassessments, many disabled people will find themselves with little option but to seek new employment.

This makes services such as Support to Work more important than ever. Disabled people looking for work should have access to support from people who understand their challenges and will work with them to come up with strategies that suit their circumstances.

We are pleased to be publishing this report at such a critical time. We hope it provides a clear example of how the right support can lead to positive employment outcomes for disabled people – and how directly involving disabled people in developing that support can make it even more effective.

Scope’s Policy and Campaigns team will be using this evidence to directly encourage Government and local authorities to consider how they will support disabled people as the pandemic continues.

The Co-Evaluation Team

Support to Work is Scope’s national employment support service for disabled people. It supports disabled people who already have transferable skills and are actively looking for work. It is a fully remote service using telephone and digital tools and runs for up to 12 weeks. Each customer works with an individual employment adviser to develop and complete a bespoke action plan.

This report evaluates Support to Work using data from the calendar year 2019. It assesses the ability of the service to achieve its intended outcomes:

• Disabled people enter employment.
• Disabled people gain the knowledge and skills they need to choose, find, and apply for jobs.
• Disabled people become confident of their capability in work.
• Disabled people know their rights when in, or looking for, employment.

The research underpinning this evaluation has a unique approach and value because it has been directed and conducted by a team of people with lived experience of disability, as well as those with professional evaluation expertise. This has allowed Scope to embed the knowledge, experience, and concerns of disabled people into an assessment of the service.

Evaluation results

Who uses Support to Work?

Around half of Support to Work customers have been out of work for fewer than 6 months. Most customers have worked before. The customer base is more ethnically diverse and younger than the national disabled population. Many customers have had negative and sometimes discriminatory employment experiences directly linked to them being disabled. This leads to many customers feeling low in confidence when joining Support to Work.

What enables people to use Support to Work?

Support to Work’s simple, friendly sign-up process enables customers to refer themselves to the service. Its wide eligibility criteria contrast with some customers’ previous experiences of employment support. The most common route for people referring themselves to the service is via Facebook, where the service is promoted. 45% of referring customers find out about the service this way. These adverts are critical places for setting expectations about the service.

The combination of phone and digital communication tools increases the service’s accessibility. It allows customers to choose their preferred communication method. Furthermore, flexibility in arranging and changing appointments supports customers with fluctuating conditions.

What stops people from using Support to Work?

Not everyone who refers to Support to Work goes on to use it. Some customers refer themselves but don’t respond to service contact. Others are looking for work experience, or would be better suited to a face-to-face service, perhaps because they’re not confident in their digital skills. Support to Work signposts these customers to alternative services. Some customers also leave the service early. For many customers we don’t know the reason for this. For those who do provide reasons, there’s a high interest in accessing the service later on.

What does Support to Work achieve for its customers?

132 customers who exited Support to Work in 2019 entered employment after using the service. This is 23% of exiting customers. The most common amount of time a customer was on the service before moving into work was 5 weeks. Most customers moving into work have been out of work for fewer than 6 months.

Customers who don’t enter directly into employment also gain from using the service. Most customers develop knowledge and skills for various aspects of the job search process, as well as the confidence to put these skills into practice. Customers’ confidence in their own capability in work also grows.

Support to Work helps many customers develop a personal strategy for if, when, and how to talk to a prospective employer about their condition or impairment. Despite this, there is limited evidence of the service’s effect on customers’ knowledge of employment rights. This may partially reflect the tools previously used to measure this.

What do customers value most about Support to Work?

Customers value:

• the skills and experience of Support to Work’s employment advisers
• the service’s format and tools
• the central focus it places on tailoring to the individual.

The acceptance, understanding, and listening skills of advisers are particularly appreciated. So is their ability to offer new perspectives and ideas. The service provides helpful structure whilst allowing flexibility to support customers with fluctuating needs. Highly tailored advice on customers’ job search processes helps individuals achieve their personally identified goals.

What’s not working well?

Disappointment can arise at times. Customers sometimes feel that they are working to a rigid or untailored action plan. This highlights that there can be communication difficulties between a customer and their employment adviser. When operating well, action plans are developed through mutual discussion and agreement. Other customers reported problems with the service’s messaging system, which is now being reviewed. Further frustration is caused by a minority of late and cancelled appointments, which can have a significant effect on a customer’s day. A final challenge is ensuring that customers have accurate and realistic expectations of the service when signing up. This report identifies recommendations for the service and details where these are already being implemented.

To what extent is Support to Work a standardised service, and how does tailoring take place within this?

The service operates to a basic skeleton model that we describe in the report. But, its strength lies in a focus on tailoring and adapting to meet individual circumstances. Positive customer experience often comes from the skills and flexibility of individual advisers to adjust how they work with different customers. Advisers use a range of approaches to give advice. This includes live and staggered feedback and practice interviews. The importance of tailoring underlines the value of investing in the interpersonal skills of employment advisers.

What is the staff experience like?

Advisers derive job satisfaction from noticing improvements in customers’ confidence and empowering customers with skills to navigate the job market independently. They enjoy working with customers who may not be able to access other services. Advisers believe that the voluntary nature of the service means customers are particularly engaged with its support. Support to Work customers choose for themselves when is the right time for them to be seeking help with finding work. At the same time, staff reported a shared frustration with the amount of time they spend setting and managing customers’ expectations.

What are the possible service gaps?

Analysis by the Department for Work and Pensions shows that disabled people are twice as likely as non-disabled people to fall out of work. Customers taking part in this research explained that it would be helpful to continue to access Support to Work’s help and advice even once they have been successful in securing work. Responding to this, from late 2020 Support to Work will begin to offer in-work support. This will be available to customers who enter employment during or shortly after using the service.

Additionally, many Support to Work customers are interested in identifying employers who have positive attitudes towards employing and supporting disabled people. There are now plans underway to introduce a jobs board element to the service. This will allow customers to search for jobs from employers who have made a commitment to inclusive practices. It will be open only to Scope customers. Customers gaining work through the jobs board will be able to remain in contact with an in-work adviser, allowing the service to monitor its success.

Summary

This evaluation finds that Support to Work offers a uniquely effective and empowering model of employment support for disabled people. The report highlights features of the service that facilitate positive outcomes. We hope it presents insights that are useful to both the direct stakeholders of the service and others working in the disability and employment world.

Introduction to Support to Work

Support to Work is Scope’s national employment support service for disabled people. It supports disabled people who already have transferable skills and are actively looking for work.

It is a fully remote service. The customer communicates with their assigned employment adviser using telephone and digital tools.

The service runs for twelve weeks. During this time, the customer works with their adviser to develop and implement a plan of action to help them move towards their employment goal. Each ‘action plan’ is unique to the individual customer, but plans frequently include guidance and recommendations on:

• improving the customer’s CV
• crafting cover letters
• conducting job searches
• completing job applications
• interview practice and performance
• if, why, when and how to tell a prospective employer about a condition or impairment
• asking for reasonable adjustments

Scheduled appointments between the customer and adviser help the customer to implement and monitor progress on their personal action plan. Appointments may be held by telephone, Skype call, Skype instant chat or email. Appointments are supplemented by extra contact where necessary. This can take place via email, phone, or through a messaging function on the customer portal. The portal is an online space unique to each customer where they can store documents, send and receive messages, and access resources that their adviser has uploaded for them. Freely available information and advice on the Scope website also supports service provision.

Support to Work aims to help disabled people become more knowledgeable and skilled in their job search. It seeks to improve customers’ belief in their own capabilities and support them to apply for jobs they are truly interested in. Support to Work customers should also become aware or be reminded of their rights as a disabled person while looking for, or in, employment. Ultimately, the service aims to support disabled people into paid employment. In this evaluation we assess the ability of Support to Work to achieve these outcomes.

Introduction to the evaluation and its approach

This co-produced evaluation consciously takes a different approach to typical assessments of service performance.

Its unique approach and particular value come from the fact it has been directed and conducted by a team of people with lived experience of disability as well as those with professional evaluation expertise. Throughout the report this team writes as ‘we’.

This approach has allowed Scope to embed the knowledge, experience, needs and concerns of disabled people into how we value our services. As a team we have asked different questions and produced insights that can come only via the perspectives gained through lived experience of disability. These have directly informed how we judge Support to Work’s success.

As such, it is our belief that we have created an analysis and a set of recommendations that maximise the possible benefit of this evaluation to disabled people.

We have actively engaged with stakeholders of the Support to Work service at different stages of the evaluation process, seeking their views on:

• what information to collect (the research priorities)
• how and with whom to share evaluation findings
• how the service might respond to findings and the co-evaluation team’s recommendations

These stakeholders include Scope staff members linked to the service, the service funder, a past customer of the service, and a representative from a Disabled People’s Organisation providing employment support. Together they formed an Evaluation Stakeholder Panel.

For the research underpinning the evaluation, we applied both qualitative and quantitative methodologies. Our sources and methods included:

• 14 individual interviews with Support to Work customers
• a focus group with 5 further Support to Work customers
• a focus group with 5 Support to Work staff members
• an interview with the Support to Work Programme Lead
• statistical analysis of pre- and post- service customer surveys
• statistical analysis of delivery data stored in the service database
• thematic analysis of free text data stored in the service database

We used the calendar year 2019 as the timeframe for our analysis.

The Appendix contains further information about methodology.

We aim to share more information on the co-production approach in a separate guide to co-produced research and evaluation at a later date.

Introduction to the report

This report presents the final findings from the evaluation research. We hope it presents insights that are useful to both the direct stakeholders of the service and others working in the disability and employment world. We find that Support to Work offers an effective and empowering model of employment support for disabled people. We aim to highlight features of the service that facilitate positive outcomes.

The report narrative starts with a focus on the characteristics and experiences of Support to Work customers and what enables them to engage with the service. We then examine where there might be barriers to people accessing Support to Work and explore how these could be addressed.

We go on to discuss the positive outcomes of the service, which include but are not limited to disabled people entering employment. This is followed by an in-depth analysis of what customers value most about the service, and conversely what’s not working so well.

Further into the report we consider to what extent Support to Work is offering a standardised service. We then zoom in on the staff experience within Support to Work and discuss how this supports the service’s goals.

Finally, we explore some possible gaps in the service and consider how these could be addressed.

Support to Work is targeted at customers who already have transferable skills but who are looking for some support with job searching, applications and interacting with a prospective employer. This contrasts with Scope’s face-to-face employment support services, which work with disabled people facing further barriers to entering employment.

The focus of Support to Work is reflected in the circumstances of people using it.

Time out of work

The length of time a customer has been out of work has recently been added as a data field in the customer management system. The available data indicates that the vast majority of customers using Support to Work have been in paid employment before.

Of the 94 customers completing Support to Work since the service started collecting this data, over half (55%, n=52) had been out of work for between 0 and 6 months – a relatively short time. As we explore in this chapter, customers in this category may be coming out of a longstanding position and need help navigating the jobs market.

The service also supports customers who have been out of work for much longer. Some customers have never been in paid employment. But, most customers joining the service have been out of work for fewer than 5 years. In future evaluations we will be able to report on these data with a larger sample size.

Demographics

Support to Work serves a diverse range of customers. The customer base is more ethnically diverse than the national disabled population, with customers of an Asian background particularly well-represented.

Support to Work customers are also younger than the national disabled population. This is partially explained by a large proportion of the national disabled population being over pension age. But, the proportion of Support to Work customers aged between 20 and 34 is substantially higher than the proportion of this age in the national disabled population.

Interestingly, the location of Support to Work customers also differs slightly from the national disabled population. The proportion of customers living in London is higher than the wider disabled population (19% of customers compared with 12% of population). Conversely, the service has fewer customers in the North West and Wales than the national picture would suggest.

There is no significant difference in the gender breakdown of Support to Work customers compared with the national disabled population.

Experiences of employment

Most of the customers we spoke to as part of the evaluation had previous employment experience, mirroring the wider customer base. We also interviewed several people coming straight out of education.

Customers’ working histories were varied. They had worked in the public and private sectors and at many levels of responsibility and management.

It was striking that despite this range of employment backgrounds, people frequently shared negative and sometimes discriminatory experiences directly linked to them being disabled.

Some had left jobs due to bad experiences when seeking reasonable adjustments following an acquired or worsening impairment. For many, leaving their previous job had been a protracted, upsetting and deeply undermining process during which they felt powerless.

In these situations, the sense of loss customers feel at no longer being able to do their job in the same way is often compounded by the poor attitude of employers and wider society. For many it’s an emotional low point in their lives.

Several people we spoke to had negative experiences of the benefits system and the demands it places on them. Some did not qualify for disability benefits and so found themselves needing to generate income at a point when it was very challenging for them to do so.

Some customers had also had poor experiences of statutory employment support services mandated as a condition of receiving health and disability related benefits.

The combination of many of these factors means that almost all evaluation participants reported having low confidence on joining Support to Work. Frequently, negative experiences in past employment or while seeking work have led directly to this decline in confidence. And our results suggest that low confidence affects customers regardless of their specific impairment or condition or how long they have been disabled for.

Customers have been made to lose confidence in themselves and their abilities, but also in employers and workplaces.

It can also be hard for customers to acknowledge what they are no longer able to do.

Many people we interviewed did not find it easy to discuss these experiences.

Implications for the service

Prior experiences greatly affect customers’ feelings when joining Support to Work and the expectations they have of it.

Some customers’ primary emotion is one of relief. Many are simply thankful to have finally found help from people who seem to understand their situation. Sometimes, this relief and hope can lead to customers having very high expectations of what the service can help them achieve. Other customers may feel sceptical and tentative, especially if they have accessed other forms of employment support before. They may find it difficult to trust that their experience will be positive. Sometimes, customers find it hard to view themselves and their abilities positively.

Regardless of the specifics, the emotional impact of experiences features heavily in many customers’ interactions with Support to Work.

This has important implications for the service. It highlights the weight of expectation that customers can place on advisers to create a positive experience and ‘turn things around’. Indeed, as we discuss in chapter 8 on the staff experience, managing customer expectations is a large part of the role for advisers.

Alongside specific advice and guidance on the job search process, advisers’ sensitive, informed words on customers’ situations are often a deciding factor in whether a customer feels that they have had a positive experience with Support to Work. Chapter 5 on what customers value about the service explores this in more depth.

Our findings also highlight how important it is that advisers understand customers’ experiences, which can have a large effect on how the customer approaches the service. Advisers’ understanding can be important in building positive customer-adviser relationships. In turn, these positive relationships are central to achieving successful outcomes for customers.

Recommendation for future practice

Consider introducing a measure of self-confidence or self-esteem at the start of a customer’s journey, so that advisers gain a clear picture of a customer’s likely need for moral support and encouragement throughout the service. This could sit alongside customers’ self-ratings as part of the Workstar assessment (see note below) or in the baseline customer survey. Consider introducing a measure of self-confidence or self-esteem at the start of a customer’s journey, so that advisers gain a clear picture of a customer’s likely need for moral support and encouragement throughout the service. This could sit alongside customers’ self-ratings as part of the Workstar assessment (see note below) or in the baseline customer survey (see Appendix for more information about the baseline customer survey). The Evaluation team is in discussion with the Programme Lead about this proposal. For more information about the baseline customer survey). The Evaluation team is in discussion with the Programme Lead about this proposal.

Our research showed that there are many factors influencing people’s ability to use and benefit from Support to Work:

• their digital skills, digital confidence and access to digital equipment
• the advertising or recommendation that led them to the service
• the sign-up information and process on the Scope website
• the tools of the service
• the culture of flexibility in the service
• customers’ individual personal circumstances and impairments or conditions

An aspect of these factors may make the service accessible or inaccessible for a particular customer. We discuss barriers to using Support to Work in the next chapter.

Here we present factors which support customers’ access and use of Support to Work.

Simple and friendly sign-up process

The customers we interviewed frequently felt positive about their route into Support to Work, including the sign-up process and their initial appointment. Customers appreciated that the service is voluntary and doesn’t require proof of disability.

Support to Work’s wide eligibility criteria, straightforward design and friendly non-judgemental approach were often in contrast to customers’ previous experiences of employment support. The stress and mistrust that people report experiencing when using support schemes offered by the Department for Work and Pensions has been documented several times.

Combination of phone and digital communication tools

It’s important to note that what may be a barrier for 1 disabled person can be the reason why Support to Work is accessible for another. This is particularly true of the phone and online based nature of the service.

Some customers found that the digital aspects of Support to Work particularly increased its accessibility for them.

We acknowledge that the service’s reliance on digital tools can also be a significant barrier to some disabled people (as we discuss in the following chapter).

Flexibility

Customers we spoke to frequently made favourable comparisons with other employment support services. These positive comments often focused on the flexibility of Support to Work. The service gives these customers some control that they felt they had previously lost or been denied. Customers didn’t feel pressured to disclose information. They could change appointments without lengthy explanation or being made to feel as if they had failed. Their individual needs were taken into account.

Because disabled people often have only one way in which something can be made accessible to them, the flexibility offered by Support to Work is important. People’s access requirements can also change and fluctuate, so the responsiveness of the service is also critical.

Emotional support

The emotional and moral support offered by Support to Work is vital to many customers and enables them to access the service and deem their experience successful. Advisers and customers generally agreed that the opportunity to build a relationship through the service supports this.

It’s also important to acknowledge that there are limits to the emotional support that Support to Work can offer. Some customers needing additional emotional support may need to be signposted to other services.

Customer stories

Customer stories in adverts or on the Scope website made some of our evaluation participants feel more able to use Support to Work. Where this was the case, it was not because they had the same impairment or condition as the storyteller, but because they had had similar experiences and felt similarly about them.

Having said this, we must also note that customer stories can sometimes have a negative effect on customers. This can occur when customers feel their experience with the programme has not been useful, pleasant or positive. In these situations, customers can sometimes internalise the root of their experience and feel that they are responsible for the service not leading to the outcome they wished for.

We mention this as we feel it may shed light on some of the barriers customers face in staying with Support to Work for the full programme length. Customers’ cumulative negative experiences and historic discrimination may cause them to feel responsible for difficulties they face in accessing support or seeking adjustments. Their expectations of accessibility and their belief that access problems will be addressed may be low. They may choose to leave the service rather than seeking adjustments.

It’s important that Support to Work advisers repeatedly check whether there are any changes to how they provide their support that would help a customer’s continued use of the service.

The following chapter considers further barriers to using Support to Work in more detail.

Chapter 2 examined enabling factors that help people access Support to Work. To provide greater clarity on who can benefit from the service, here we present insight on people who leave the service early or who refer themselves but are not accepted onto the service.

Reasons for people not starting on the service

In 2019 Support to Work received referrals from 1958 unique customers. Of these customers, 621 started using the service. Some of these customers re-used the service within the calendar year. 1362 referrals to the service did not result in a customer joining the service.9

A data field recording reasons for customers not starting the service was added to the service database in mid-2019. Accordingly, there are quantified ‘did not start’ reasons for 259 customer cases in 2019. Of these, the most common is that the referring individual disengaged by failing to return phone calls or emails.

The Support to Work Programme Lead also performed a manual analysis on the full number of referrals received across 2019. This confirms the analysis performed on the partial data from the service database. The most common reason for referring individuals not starting Support to Work is a lack of contact. This can be a failure to respond to repeated messages or non-attendance at an initial appointment.

The Programme Lead’s analysis also shows that the second most common reason for non-starts is that the service is not suitable for the customer. Within this overarching reason, referring individuals may be:

• too young
• not disabled
• not looking for paid work (but rather work experience or volunteering)
• unaware that someone had referred them
• wanting a face-to-face service
• wanting to work with a recruitment agency
• wanting someone to write applications on their behalf

Finally, advisers deem some referrals (273 in 2019) ineligible for the service after speaking to them during an initial appointment. This ineligibility can be because:

• the customer has had a change of circumstances since their initial referral
• the self-assessment they provide as part of the Workstar indicates that the service is not right for them at that time.

These combined reasons highlight that some people referring themselves to Support to Work have inaccurate ideas or expectations of what the service offers. Our qualitative research suggested that not everyone gains a clear picture of the service from its promotional material:

Advisers also reported that they spend a lot of time managing customers’ expectations (as we discuss in Chapter 8 on the staff experience). Service expectations can also influence early exits from the service.

Reasons for leaving the service early

Some customers leave Support to Work before they have received the full 12 weeks of support. In some cases this is due to the customer gaining employment (see Chapter 4 for more detail on positive outcomes). Further reasons for exiting the service early are recorded by advisers in the service database as free text. This data illuminates barriers and external circumstances contributing to early exits. A range of reasons exist for the 268 cases of the service ending early in 2019.

Customer’s lack of communication or engagement

By far the most common reason recorded for early exits is not hearing from customers, echoing the main reason for customers that don’t start the service.

“[Customer] has been exited from the service as [they have] not responded to emails or calls. [They have] not attended any appointments beyond [their] second appointment.” 

This particular free text record also reflects the fact that of the customers who leave the service early, the majority do so before their third appointment:

When customers simply stop responding, it is hard to judge whether Support to Work could do something differently to support them. It’s possible that some disengagement is due to customers being unhappy with the service. Our qualitative research sheds light on some of the main causes of dissatisfaction within the service; see Chapter 6. But, it’s also possible that lack of commitment or engagement is caused by one of the following additional reasons for customers leaving the service early.

The customer no longer requires the service

Some customers leaving early choose to do so as they feel they no longer need support with finding employment.

“No longer needs support from the service as [they feel they] can move forward independently”

This chimes with comments from advisers who said they will close a customer’s case if they feel they are equipped with skills:

This demonstrates that Support to Work can provide effective support within a rapid timeframe.

The service is not appropriate for the customer

Some customers leave Support to Work because they require in-person support or lack the digital confidence or skills to access the service fully.

“[Customer] had not uploaded [their] CV and said that [they] need support with this as [they] cannot do it [themselves]. We discussed our service in more detail and agreed that it is not the right service for [the customer] and that a face-to-face service would be more beneficial.”

Recommendation for future practice: The service should continue to maintain a list of sources of face-to-face support, so that onward signposting is possible for customers who would benefit from this. Support to Work may also want to consider maintaining information and resources on where customers can access support with using digital resources. Digital exclusion remains a widespread problem within the disabled population, with 42% of disabled people having a Low or Very Low digital engagement as measured by the UK Consumer Digital Index 2020. This is compounded by the fact that disabled people are 40% less likely to have received digital skills support from their workplace

Other customers ask to be exited from the service after realising that the service cannot provide things they were expecting, such as job brokerage or curated lists of vacancies.

“Customer unhappy I did not send [them] a list of jobs [they] can do based off [their] skills and experience.”

This was reflected in customer interviews, where we found that disappointed customers often started with an expectation of the service that went beyond what it offers. This emphasises the importance of clear, unambiguous marketing and supporting information about the service before someone becomes a customer.

Identified Action: We have recommended that the content explaining Support to Work on the Scope website is refined to highlight that the service cannot currently:

• offer job brokerage, where an adviser will liaise with a potential employer with the aim of securing the customer an interview, work trial or job
• source suitable vacancies for individual customers (other than while demonstrating how to conduct an independent job search)
• provide access to an exclusive list of ‘approved’ disability-friendly employers
• write a customer’s CV on their behalf, using information that the customer shares with their adviser

This content is being updated. We also recommend that the customer coordinator and advisers reiterate the service boundaries when a customer has their initial assessment. Taken together these measures should reduce:

1. the number of people referring to the service but not commencing it
2. the number of people leaving the service early

Outside commitments

Difficulty combining Support to Work with study, voluntary work or other services is another common reason for customers leaving early. This emphasises the importance of advisers supporting customers to understand the level of commitment necessary for full engagement with the service.

Health

A significant number of exiting customers cite ongoing health concerns as their reason for leaving the service early. This is a reminder that many disabled people have fluctuating conditions that can unexpectedly affect their ability to look for work. However, it may also represent the pressure that a lot of disabled people feel to find employment when the support they receive from the state is insufficient to support them, as one of our evaluation participants explained.

Personal reasons

Finally, a wide range of personal issues are recorded as reasons for leaving the service early.

Likelihood of re-engaging

Positively, despite some customers asking to exit the service early, the free text records also show that many would access the service again. In many instances, advisers actively suggest and encourage this future engagement, enabling customers to feel welcome to return to the service when ready. We discuss the value of customers using the service at the right time for them in Chapter 8.

Data Quality

It is important to note that free text entries in the service database differ widely in the amount of detail recorded, depending on the individual contributor. This limits understanding of the main barriers to engaging fully with Support to Work. Additionally, the exclusive use of free text to capture reasons means that their aggregation and analysis take up a lot of time.

Identified Action: Using this evaluation data, the Evaluation team have proposed a set of core categories for early service exit reasons. The Programme Lead has added these categories as a reportable field in the service database, allowing exit reasons to be recorded in a standard format. The free text box that already exists for recording exit reasons will be maintained so that categorised data can be supplemented with further information where appropriate.

Identified Action: Support to Work is now recruiting a second customer coordinator who will be responsible for following up with Support to Work customers leaving the service. They will do this whether customers are exiting early or their time with the service has expired. This coordinator will record accurate destinations of these customers. We propose that they should also consistently record reasons for customers leaving the service. This should lead to greater understanding of the barriers that customers often face in accessing the service.

One important aim of the evaluation was to explore what positive changes happen for customers as a result of using Support to Work. Our evaluation questions included a focus on outcomes that the service intends to achieve.

These include:

• disabled people entering employment
• disabled people gaining the knowledge and skills they need to choose, find and apply for jobs
• disabled people becoming confident of their capability in work
• disabled people knowing their rights when in, or looking for, employment

This chapter discusses these and related outcomes. We present the different outcomes in turn.

Disabled people entering employment

The ultimate goal of the service is to support disabled people into employment.

The service database stores information about whether a Support to Work customer enters employment when they leave the service. In 2019, 23% or 132 out of 578 exiting customers moved into work.

Advisers also contact customers who enter employment 13 weeks after they have started work, to gather data on whether they are still in employment. In the same period, 148 customers who had moved into work reached the 13-week milestone. Of these, Scope was able to reach 73 customers. 51% could not be contacted.

Of the customers we were able to reach, 75% were still in work after 13 weeks of employment.

We are confident that this is a representative picture of all customers reaching 13 weeks of employment, because when we analysed the data by different customer characteristics, the results were broadly the same.

The service does not currently monitor the employment status of all customers 13 weeks after they have left the service, only those who exit straight into employment.

Recommendation for future practice: The recruitment of a second customer coordinator, as mentioned in Chapter 3, should allow Support to Work to better monitor the eventual employment outcomes of all customers, not just those exiting straight into employment. Some customers may not find work whilst they are using the service but do so soon afterwards. Capturing this data would generate a more accurate picture of how many people enter employment after using Support to Work.

What kinds of jobs do people go into?

For customers who entered work in 2019, almost 3 quarters (73%) went into jobs of over 16 hours a week. 18% entered work of 8 to 16 hours, and 9% less than 8 hours per week.

The types of jobs that customers enter vary. Using the Office for National Statistics Standard Occupational Classification 2010 (SOC2010) skill levels, the most common skill level of jobs that customers enter is Level 2. 51% of customers enter jobs at this level.

Examples of jobs at different ONS Occupational Skill Levels

1. Cleaner, catering assistant, hotel porter
2. Carer, machine operator, administrative assistant
3. Paralegal, electrician
4. Corporate manager, doctor, lecturer

When do people enter work?

There is large variation in when customers start a job. The most common amount of time someone has used Support to Work before moving into work is 5 weeks. This may be explained by the responsive nature of Support to Work that allows customers to receive tailored support for specific tasks, such as an upcoming interview.

Are customers satisfied with the roles they go into, and do they keep their jobs?

The quantitative data presented above shows how many customers enter employment after using Support to Work. But, it cannot speak to whether customers are happy with their new job roles. Through interviews and analysis of case notes in the service database, we heard from participants who were pleased with their job and journey.

In contrast, one customer we interviewed reported that they did not continue with their new role.

The case notes also revealed 2, 2019 customers, who had not stayed in employment.

The experience of these customers demonstrates that not all employment is sustained long term. A range of interacting factors can affect the longevity of job outcomes. Some of these factors may not be within the direct influence of Support to Work in its current form.

Participant 6 had a change to their health that affected their ability to stay in work. The customer from the case notes excerpt didn’t thrive in their work environment. These cases may or may not have been foreseeable. The situations could have potentially been worked around with the help of an understanding employer.

Regardless of the exact dynamics at play, the result was that these customers left recently gained employment. This is common. Analysis by the Department for Work and Pensions shows that disabled people are twice as likely as non-disabled people to fall out of work.

Further into this report, in Chapter 9, we explore the merits of customers maintaining contact with Support to Work even after entering employment. This could possibly help to reduce the incidence of people leaving jobs soon after gaining them.

What are some of the factors influencing employment outcomes?

Our research and analysis identified some factors that can influence whether someone enters work. This insight increases understanding of the service and may inform how we provide the service in future.

Time out of work

Deeper analysis of job outcomes showed that the number of customers moving into work varies substantially depending on the length of time the customer has been unemployed for. Figure 4.4 shows that fewer long-term unemployed customers enter work.

Most customers moving into work have only recently left a job. This suggests that the amount of time someone has been out of work is an important factor in influencing whether they find employment whilst using Support to Work.

Customer confidence and perception of self at service entry

We also explored how customers’ baseline Workstar measures related to their employment outcomes.

For every Workstar category, the baseline score was higher for customers who entered employment compared to customers who didn’t. This suggests that customers who start the service feeling more prepared for work are more likely to enter employment.

But, changes between the baseline and exit readings of all Workstar categories except ‘Basic skills’ also had a significant relationship with whether a customer moved into work. People who moved into work showed a larger change between their baseline and endline readings than those who didn’t. This demonstrates the contribution that Support to Work makes in equipping customers to find and enter work.

Wider circumstances

Customer interviews and the focus group highlighted some of the wider circumstances that can affect customers’ journeys into employment. These circumstances must be considered when judging what constitutes a successful outcome of Support to Work. The absence of eventual employment does not necessarily mean that the service has not been of value to the individual. There may be wider, structural barriers to employment outcomes.

Support to Work cannot directly influence:

• the attitudes of individual employers towards disability, age, or any other protected characteristic
• the accessibility of individual application processes
• local availability of jobs, public transport and childcare
• the boundaries of permitted work when receiving certain benefits

With or without Support to Work, these factors may have a large influence on how possible it is for a disabled person to enter employment.

Within these constraints, we heard from customers and staff alike that the service generates many positive outcomes aside from entering employment.

We explore these in the rest of the chapter.

Disabled people gaining the knowledge and skills they need to choose, find and apply for jobs

Support to Work helps customers gain or consolidate the knowledge and skills they need to find employment. Customers can apply these skills to their job search both during and after using Support to Work.

Data from baseline and endline customer surveys shows that customer confidence for job interviews, application forms, writing cover letters and writing a CV all significantly improve when using Support to Work (see the appendix for further detail on the methodology used in this analysis). This is shown in Figure 4.5. Responses are coded on a scale where 1.0 equals not at all confident and 5.0 equals very confident.

Our qualitative data also offers rich evidence for this improvement in knowledge and skills. This can occur regardless of whether a customer enters employment.

8 participants reported greater knowledge and confidence in CV development. Customers spoke about receiving advice on content, formatting, and updating.

3 participants also spoke about new knowledge and skills for finding vacancies.

One participant specifically mentioned application forms and how they’ve used new skills independently after accessing Support to Work.

Customers reported interview skills as a particularly useful outcome of using Support to Work, with 5 participants reflecting on this especially. Holding a mock interview was particularly valued.

In Chapter 7 we go into more detail about features of the service that link to these knowledge and skills outcomes.

Disabled people becoming confident of their capability in work

In our research, customers and staff referred frequently to changes in self confidence, mindset and how customers see themselves through accessing Support to Work.

Improvements in confidence and self-belief were some of the strongest outcomes reported by evaluation participants.

The customer baseline and endline surveys also demonstrated that, on average, customers gain confidence in their capability to perform well in a job after using Support to Work. On a standardised scale of 1.0 to 5.0, on average customers improved in their confidence from 3.69 to 3.87. There was an improvement for every question asked.

This is further reflected in survey data showing that customers apply for more jobs that are ‘interesting’ and ‘a good step for my career’ after accessing the service.

Confidence links closely to a sense of empowerment. This was a theme that both customers and staff referred to in our research. Empowerment means that someone has gained the power to do something for themselves. It comes from a combination of knowledge and confidence.

3 participants talked about empowerment as a success of the service.

Processes of empowerment ensure that customers aren’t just learning but feel able to apply new knowledge and skills independently, both during and after the service.

Appropriate tailoring of tasks and action plans is part of what encourages empowerment. Different customers need varying levels of encouragement, support, and guidance before they can move towards independent actions.

This underlines the importance of realistic and supportive action plans, which we talk more about in Chapters 5 and 7.

Disabled people knowing their rights when in, or looking for, employment

Support to Work aims to improve customers’ knowledge of their employment rights as a disabled person. If customers know what rights they have and how they can respond when their rights are infringed, they may be less likely to have to leave or change jobs – as many Support to Work customers have already had to do.

Not all customers we interviewed actively wanted advice on their rights. Some customers were already confident in their knowledge and application of rights, while others felt it wasn’t relevant to them at the time.

But, other customers clearly appreciated being able to talk through not only the details of equalities law, but also practical ways to discuss disability with an employer.

4 customers specifically mentioned their adviser helping them to develop a clear, individual strategy for disclosing their impairment or condition and asking for reasonable adjustments. Rather than an improvement in knowledge of rights, this reflects more a confidence in using rights to one’s practical advantage:

This is echoed in customer survey data showing a small improvement in customers’ confidence to ask a manager for reasonable adjustments after using Support to Work. Advisers’ understanding of disability and anticipation of possible needs that we discuss in Chapter 5 helps advisers present useful ideas to customers.

One customer we interviewed felt that Support to Work didn’t help them learn about their rights.

There was no significant change in the number of customers answering factual employment rights questions correctly in the customer survey, and so there is no quantitative evidence for this outcome.

We have since updated the questions used to measure knowledge of rights. The previous measure was much simpler and may have been too blunt to capture changes. The updated measure should allow future evaluations to contain more detailed analysis of knowledge in employment rights.

One participant felt they may want to come back to Scope for additional advice on rights in the future, perhaps when in work. We discuss the demand for in-work support in Chapter 9.

Our customer interviews and focus group shed light on features of Support to Work that customers value highly. These elements all contribute to customer satisfaction and sustained engagement with the service. They are likely to be associated with greater probability of positive employment outcomes.

We have organised the features into 3 broad categories:

• The skills and experience of the Support to Work advisers.
• The tools and underlying format of the service.
• The central focus on tailoring to the individual.

These features tend to support and overlap with each other when the service is working optimally.

The skills and experience of the Support to Work advisers

At the heart of the service sits the working relationship between a customer and employment adviser over a twelve-week period. This makes adviser skills and experience critical to customers’ successes. Customers place great value on the following.

Listening skills

Satisfied customers feel that their adviser makes an effort to truly hear and understand their unique situation. Understanding someone’s circumstances involves listening to their hopes, concerns, experience and skills. A significant number of evaluation participants praised the listening skills of their adviser.

Advisers take customers’ experiences on board through this attention to listening. They also ask sensitive questions to uncover further information when appropriate. This helps to build an even broader picture of a customer’s circumstances.

The offering of a new perspective and new ideas

The genuine effort to listen to and grasp a customer’s circumstances helps advisers reframe someone’s situation. Advisers apply what they have learned from conversations. They highlight positives that the customer may not have considered before, or which they currently find it hard to focus on.

A new angle on things can transform how customers feel about their job search and the possibility of gaining work. It can be particularly powerful for customers who are feeling constrained in their options and who have lost confidence in themselves. As discussed in Chapter 1, this might be the case if the customer has had to leave their previous role due to changes in their condition or impairment, or because of an employer’s negative response to such changes.

Such customers talk about the change in outlook that their adviser helps them reach:

Many customers also appreciate their adviser presenting novel ideas. When Support to Work is working at its best, advisers are highly tuned in to someone’s experience, aspirations and apprehensions. They combine this understanding with their employment market knowledge to suggest alternative ways of reaching goals. New ideas might directly relate to a job search or could be more widely supportive of someone’s situation. As an example, here an adviser helps a customer access the disability benefits she was entitled to:

Customers highlight how ideas are presented as helpful suggestions rather than directions on the ‘right’ and ‘wrong’ ways of doing things. We revisit this observation later in this chapter under ‘Central focus on tailoring to the individual’.

Acceptance and understanding

Customers also commend how Support to Work advisers demonstrate acceptance and understanding.

Many disabled people using Support to Work have encountered alternative employment programmes elsewhere. They may also have applied for state support as a disabled person. Customers emphasised the contrast between other forms of support and Support to Work. The contrast is palpable when entering the service, even before building a relationship with an adviser. Support to Work’s open eligibility criteria contrast positively with the need to provide proof of disability or financial hardship before using other sources of support.

Then, when working with their adviser, customers often feel relief that they understand the challenges that disabled jobseekers can face. Advisers grasp that many of these challenges arise from social barriers. They work from the frame of the social rather than medical model of disability. The social model says that people are disabled by barriers in society, not by their impairment or difference.

More information about the social model of disability on Scope's website.

Some customers may not be aware of the social model before using Support to Work. Advisers’ understanding of this perspective is another important way that they can help a customer positively reframe their situation.

Over time, advisers have built up knowledge and understanding of a broad range of disabled people’s challenges in finding work. This means that they can often anticipate likely needs in advance.

Although advisers should always listen and respond to each person’s unique situation, this ability to anticipate needs means advisers can be ready to support someone with common difficulties. Advisers with lived experience of disability also contribute a unique empathy with customers’ challenges and deepen the level of understanding within the service. In 2019, half of the Support to Work team were people with lived experience of disability.

Finally, simply acknowledging someone’s challenges can help customers feel validated:

Advisers’ commitment to acceptance and understanding overlaps with the service’s focus on individually tailored action plans. We discuss this in the last part of this chapter.

The tools and underlying format of the service

Customers also value how Support to Work is structured and what this means when working with their adviser.

Not everyone thrives using every part of the service structure and supporting technology. So, advisers adapt to work in the way that suits each customer best. We discuss this more in Chapter 7. Here are some of the structural features of the service that customers appreciate.

The online customer portal

Customers who feel able to engage with their online portal find it useful as a central place to share, store and find all documents and resources relevant to their job search. Customers looking for specific pieces of information (for example, on disability-friendly employers) can access resources that their adviser uploads for them.

Other customers particularly like the portal for its messaging system, especially when they feel too anxious to make a phone call.

It’s important to note that not all customers find the portal useful. We cover this in more detail in chapter 6.

First-hand practice of workplace skills

3 customers we interviewed spoke explicitly about the benefit of repeatedly speaking to an adviser over the phone. Telephone practice supports customers to feel more confident and prepared for telephone interviews, which are the first step in many recruitment processes.

Further, 6 customers highlighted the value of holding a mock or practice telephone job interview with their adviser. This closely mimics a process that many customers will encounter in their job search.

Structure of weekly appointments – but with the flexibility to rearrange

As we mentioned in Chapter 2, the fluctuating nature of many disabled people’s conditions or impairments – for example those with chronic illness or pain – can mean that they may need to rearrange previously agreed appointments or tasks. Several of the people we interviewed said how much they appreciated being able to do this without fear of it meaning they would miss out or be sanctioned. Customers said it was generally possible to rearrange or adapt appointments to suit them:

This flexibility sits within the service’s commitment to regular contact with customers. Continuity and regularity help customers break down large goals into achievable tasks to complete and review each week.

The central focus on tailoring to the individual

When Support to Work excels, advisers’ skills and the underlying service tools combine to produce a fully tailored experience. Customers receive exactly the direction, advice or support they need for their unique situation. They are also supported to have an active role in planning the tasks that will help them towards their goals.

Tailored job search support can come in a variety of forms. Some of the most valued aspects of good personalised design are covered below.

Bespoke CV advice

Multiple customers praised the level of detail in the CV review and development process. Advisers make specific suggestions on what to improve, applying their understanding of the customer.

Highly tailored interview preparation

Once customers have identified specific job roles they would like to apply for, advisers can help them get ready for the interview process. They work with customers to prepare detailed answers to common interview questions. Customers often then practise these answers in mock interviews.

As Participant 4 highlights, tailoring encompasses not only the specific content of support, but the timing and tone behind it too. Tailoring can be just as much about moral support as about bespoke employability advice.

Advisers’ balance between commending and critiquing

Several customers we interviewed highlighted how their adviser generally found the right tone and balance between positive and constructive feedback. We see this as an element of tailoring, because getting the balance correct relies on advisers’ understanding of a customer’s confidence, resilience and wellbeing levels.

Indeed, not every customer feels that their adviser gets this right. This is a significant challenge for the service (see Chapter 6). If the balance between positive and constructive feedback isn't quite right, it can have a damaging effect on customers’ ongoing confidence. But, when advisers judge their tone well, customers appreciate it a great deal. 7 customers picked this out as something they particularly valued about the service.

Role of the action plan in tailoring

Focussing on the areas that customers most need help with relies on advisers and customers working together to develop an appropriate action plan.

An action plan is a broad strategy that addresses an individual customer’s priorities in their search for employment. It might cover things like:

• updating a CV
• practising application forms
• discussing how and when to talk to a potential employer about an impairment or condition
• exploring places to look for work

Advisers and customers should discuss and mutually agree the action plan in a customer’s initial appointment. This means both parties are aware of the customer's goals for the duration of Support to Work, so that this time is best spent.

Good action plan development occurs when customers feel that the planned goals and associated tasks have been agreed in partnership. Activities should feel achievable from week to week.

We cannot overstate the importance of developing action plans by discussion and mutual agreement. In Chapter 6 we explore where the biggest breakdowns in Support to Work occur. This is often when a lack of quality communication between advisers and customers leads to disappointment with action plans and the service more generally.

Our evaluation questions included a focus on customer frustrations and negative experiences. This helped us generate a picture of where problems can arise in Support to Work. 4 of the 19 customers we spoke to felt their experience with the service was more negative than positive. Several other evaluation participants had some isolated negative feedback.

Likelihood of giving feedback

We want to mention first that when evaluation participants expressed a difficult experience with Support to Work, we offered them the chance to raise a complaint or engage with the Programme Lead to address the problem.

Several participants felt uncomfortable at the prospect of formally raising concerns.

This discomfort may partially explain why some customers leaving Support to Work early choose simply to disengage with the service rather than trying to raise concerns. It also underlines why evaluation is important. It gives customers the opportunity to share their experience through alternative channels – in this case, with researchers with lived experience of disability.

When participants did want to take their feedback further, the Programme Lead responded to comments quickly.

We now discuss the main sources of customers’ negative experiences with the service.

Rigid or untailored action plans

Chapter 5 discussed how helpful action plans can be when they are tailored to the individual customer. In contrast, customers who were dissatisfied with their action plans felt that they were rigid and prescribed. 2 customers described their action plans feeling like a ‘tick box’ exercise.

This feeling may arise because of the importance of action plans as a tool for advisers to manage their caseload. The Programme Lead confirmed that action plans are a useful means by which advisers can track progress with each individual customer.

But as we highlighted in Chapter 5, strong action plans rely on active discussion and mutual agreement between the adviser and customer. Ample time for dialogue encourages the rapport and information sharing that promotes accurate tailoring.

If a customer and adviser have not built this rapport, customers may feel unable to question and challenge their path through the service. This reduces an adviser’s ability to tailor an action plan.

Indeed, a lack of a solid working relationship and associated communication difficulties appear to play a significant role in contributing to customer dissatisfaction.

2 customers’ disappointment with action plans was further exacerbated by unhelpful suggestions from their adviser. One of these customers also reported that their adviser could not recall the actions they had set in a previous appointment and that their adviser wasn’t completing their agreed actions.

This experience highlights the importance of accurately recording conversations and agreed actions. Good notes help avoid the frustrating situation experienced by this customer.

Notes also prevent customers having to repeat themselves. Talking through needs or challenges on multiple occasions can be stressful and reduce trust in the service. This was the case for one customer we interviewed, who found that relevant information they had shared had not been noted or acknowledged.

Accurate written records also help different advisers communicate with an individual customer. This might be necessary in cases of staff sickness.

Recommendation: Our findings highlight the importance of good working communication between and a customer and adviser. Problems can arise when this is absent. We suggest that Support to Work offers customers a way to express relatively early in the service how they feel about working with their assigned adviser. This could take the form of an automated email invitation from the Programme Lead or a Scope employee outside the service, giving customers the opportunity to provide feedback. This would improve understanding of early customer experience and create an easier route for customers to express potential issues at an early stage. This feedback mechanism could support continued engagement with the service and potentially reduce the number of customers leaving the service early, as explored in Chapter 3.

We also refer to Chapters 5 and 7 for further recommendations and best practice in the creation of action plans.

Frustration with portal messaging system

The 2018 evaluation of Support to Work highlighted some difficulties faced by customers using a mobile device to access their online portal. This was repeated in our 2019 data.

Specifically, 5 out of 19 participants reported frustrations with the portal messaging system. These frustrations mainly related to difficulty in sending and viewing messages.

Related to this, one customer suggested that the portal was inaccessible for people with learning difficulties and dyslexia.

In the following chapter, we explain how advisers sometimes avoid using the portal when a customer has identified that accessing it is difficult for them. We also have raised the issues with the portal messaging system with the Programme Lead.

Identified Action: When we reported customers’ difficulties using the portal messaging system, the Programme Lead confirmed that this had already been identified as a service issue. Scope’s Digital team are now investigating the source of the problems and working to implement a solution. In the meantime, customers can send and receive emails from Support to Work instead. Advisers should also continue to use this approach where it better meets a customer’s accessibility needs.

Late, cancelled or short appointments

Another area where customers had negative experiences was when their appointments had been changed, cancelled, or shortened. This was especially the case when changes or cancellations happened at short notice.

Only 3 out of 19 customers raised these concerns, but in these cases it led to customers forming a negative view of Support to Work.

As we mention in Chapter 5, the flexibility for customers to change appointments is highly valued. However, this does not necessarily mean that customers are able to accommodate changes to appointments made by advisers.

The wider context is important here. A call with a Support to Work adviser may:

• be the important event in someone’s day
• consume a large percentage of someone’s energy for that day
• involve a lot of additional planning and preparation on the customer’s part
• induce anxiety

These factors heighten the effect of late or cancelled appointments. Customers can end up feeling unvalued and demotivated.

Short calls can also lead to customers feeling frustrated.

This once again links to the need for appointment time to be protected for ample conversation between adviser and customer so that mutual understanding can grow.

The adviser team explained that the length of appointments can vary according to several factors. Appointments are booked in 60 minute slots, but advisers and the Programme Lead explained that calls can be much shorter if a customer:

• is not particularly chatty
• is still working on the same task that was set during the previous appointment

Some calls also extend over an hour. This may explain why some customers reported that calls from their adviser were running late.

Although call length can be disappointing for some customers, quantitative analysis of call length and customer outcomes found no relationship between the two.

Unmet expectations

A few customers we spoke to felt that their expectations of Support to Work had not been met.

In Chapter 1, we highlight how prior experiences, discrimination and the scarcity of employment services for disabled people may lead to customers having unrealistic expectations of the service. And as we discuss in Chapter 8, staff also raised this as an important issue.

We interviewed 2 customers who had not fully realised what Support to Work offers. This led to significant disappointment.

As we mentioned in Chapter 3, it’s vital that all customers understand as early as possible what the service will, and will not, provide. This information must be not just available, but also accessible to each individual. This means it must be presented in a range of ways. We discuss the role of service advertising in the next chapter.

Misguided expectations can lead to dissatisfaction, early exits, and wasted time and resources. From the best of intentions, it may be tempting for advisers to be emphatically positive in an initial appointment, but it is in nobody’s best interests.

An ongoing commitment to helping customers maintain realistic expectations of the service will maximise positive experiences for both advisers and customers. Several of this report’s recommendations relate to managing expectations.

Effect of negative experiences

While most customers we spoke to as part of the evaluation experienced a clear increase in confidence, some interviews also revealed that poor experiences with Support to Work have the power to negatively affect confidence and perception of self. This was the case for 4 out of 19 participants. Negative experiences are often related to the issues discussed in this chapter.

Engaging in Support to Work can also evoke difficult emotions.

Accessing the service often coincides with:

• reflecting on previous experiences that may have been difficult
• coming to terms with acquiring an impairment, or experiencing a worsening in a condition
• the feeling of having exhausted other options

As we discuss in Chapter 1, previous experiences can weigh heavily on customers. When customers also have a negative experience with Support to Work, the adverse effect on confidence can be magnified. Future evaluations could explore the effect of customers’ experiences on their interaction with Support to Work in more depth. This could help the service anticipate helpful actions.

The recommendations and actions identified in this chapter and throughout the report suggest ways to minimise potential negative effect of the service.

The Evaluation Stakeholder Panel put forward several evaluation questions related to possible variability in how Support to Work is provided. Stakeholders were interested in:

• how far the service follows a standard process
• whether certain approaches to delivery have an observable effect on outcomes and thus could be considered best practice

A certain level of standardisation is necessary to describe and manage the service. Some standardisation should also guarantee that customers receive a consistent level of service, regardless of when or how they access the service. It is also relevant to applying the service model within other settings.

As we’ve identified in the previous 2 chapters, positive customer experience can often be attributed to the skills and flexibility of individual advisers to adjust how they work with different customers. Indeed, the service’s focus on tailoring and adapting to meet individual circumstances is a central part of the service model.

In this chapter we examine the main elements of a customer’s interaction with Support to Work and describe:

• where there’s more standardisation
• where there’s more variability
• the implication of this for customer experience

We found no significant relationship between number of appointments or time spent in appointments with customer outcomes. So we examine and discuss variability mostly qualitatively.

The Support to Work customer journey

The following list summarises a customer’s main touchpoints with Support to Work. It assumes that the customer is fully eligible for the service.

1. Customer reads or hears about the service.
2. Customer finds out more from the Scope website or Support to Work phone number.
3. Customer refers themselves to the service.
4. Customer coordinator gathers further information for the customer.
5. Customer has initial assessment with their assigned employment adviser and discusses needs and goals.
6. Customer and adviser work together to create an action plan for the customer's time with the service.
7. Adviser encourages customer to use their online portal to follow their action plan and upload documents.
8. Customer and adviser book in further one to one appointments relating to agreed actions.
9. Customer and adviser continue to upload resources and documents to the online portal.
10. Further support needs were identified and support provided as necessary.
11. Support continues until customer enters employment or 12 weeks is reached.
12. Adviser contacts customers in work 13 weeks later to find out customer's circumstances.

This shows that a large proportion of a customer’s experience depends on how the customer and adviser define the customer’s support needs and appropriate actions. This naturally varies between individual customers.

However, during the staff focus group, advisers expressed that they would expect a customer leaving the service to be equipped with the skills and confidence for:

• looking for vacancies
• writing a CV
• writing a cover letter
• completing an application form
• interview performance

Advisers also suggested they would ensure customers had a personal strategy for if and when to talk to an employer about their condition or impairment.

These comments imply a standardised approach to service content, even if not to delivery methods.

Customers and staff participating in the evaluation indicated that a review of the customer’s CV is often the first substantial part of a customer’s action plan. Advisers can be reluctant to support a customer with other items before first reviewing their CV. This is probably because a CV gives the adviser a lot of context of someone’s work experience and how they are currently portraying themselves in a job search. It makes a useful starting point to identify other possible actions.

2 customers that we spoke to felt highly frustrated that they were asked to submit their CV at the start of the process. They had wanted to focus on other elements of their job search and had been told that this would be possible. Their experiences relate to the sense of ‘tick boxes’ discussed in Chapter 6, as well as customers’ expectations of the service.

Understandably, some customers interpret a tailored action plan to mean that they can move straight onto interview preparation or talking about reasonable adjustments.

The Programme Lead confirmed that reviewing a customer’s CV should not be an automatic first step in an action plan. The adviser and customer should discuss the customer’s priorities when setting initial actions. The customer should have the opportunity to guide their starting point and first tasks.

Elements of the customer journey: how standardised?

Finding out about the service and referring to the service

The most common route for people referring themselves to Support to Work is via Facebook, where the service is promoted through posts and paid-for adverts. 45% of people who refer themselves find out about the service this way. This is true of customers in all demographics. The next most common route is through a search engine.

Facebook adverts and search engine listings are thus important places for setting customers’ expectations about the service.

Identified action: As discussed in Chapter 3, we have suggested clarifications to the information about Support to Work available on the Scope website. These clarifications can also be applied to external marketing such as Facebook adverts. We have suggested these improvements to the Scope Marketing team so that external promotion sets the right expectations early in the customer journey.

Once on the Scope website, a customer can read further standardised information about the service. This may be supplemented by an email or phone conversation with a member of staff.

All referring customers must sign up for a Scope account and provide certain information about themselves via a standard online registration form.

Speaking to the customer coordinator

After completing a registration form, customers speak to the Support to Work customer coordinator. The coordinator gathers more information about the customer in preparation for their assessment with a named employment adviser. The coordinator also reiterates what the service can and can’t provide, to further set customer expectations.

Initial assessment with assigned adviser

The initial assessment is standardised in that each customer completes a Workstar appraisal in conversation with their adviser.

A customer rates themselves on different measures relating to their job search. Advisers use these scores against a common framework to assess whether Support to Work is ultimately suitable for the individual. People for whom the service is unlikely to be helpful are signposted to alternative Scope or external services.

The conversation underpinning the Workstar assessment also gives the employment adviser contextual information about the customer’s previous experiences and their main barriers to finding employment. It’s a central tool for helping the customer and adviser identify a suitable action plan.

As touched on in the previous chapter, there seems to be variation in how much advisers deviate from what could be considered a ‘template’ action plan.

Creation of the action plan

The action plan guides how an adviser and customer progress through the service. It sets out tasks for the customer to complete between appointments which the adviser and customer can then review together.

They are thus a part of the service where standardisation is not encouraged.

In some cases this works very effectively, as discussed in Chapter 5. However, as noted in Chapter 6, there can sometimes be too much rigidity in action plans. Customers who are low in confidence and have not yet built a relationship with their adviser may not feel able to challenge a proposed plan.

Although there are common elements to many individuals’ job searches, advisers should always allow flexibility in the content and speed of action plans. They should ensure a customer has ample opportunity to make their own suggestions or counterproposals. Advisers should proactively check whether customers are satisfied with action plans, particularly as some customers may feel uncomfortable about directly challenging their adviser.

Identified action: After we informed the Support to Work Programme Lead about some customers’ dissatisfaction with their action plan, the Lead ran an exercise with the adviser team. Advisers scrutinised and discussed what makes a strong, customer-centred action plan and refreshed their approach to action plan creation. This should lead to greater customer satisfaction with action plans resulting in better engagement with and completion of actions.

Use of customer portal

Advisers encourage customers to access their online portal between appointments. The portal is where customers can review their action plan, read guidance that the adviser has uploaded, and update their individual CV and cover letters.

As discussed in the previous chapter, the portal is not accessible to all customers. One customer we spoke to only had access to a computer at their local library. They found using the portal on their phone difficult, and so only had infrequent access to it when they could go to the library. To respond to the situation, their adviser sent information and advice in the body of an email which could be read on a phone, rather than uploading it to the portal.

This example once again illustrates the importance of flexibility and the ability to deviate from the standard customer journey.

Content supporting a job search

The table below summarises some of the service’s main approaches to various elements of a job search, as established by the customer interviews and focus group. We do not have sufficient data to say that any particular approach could be considered ‘best practice’ for generating positive outcomes.

We wish to note that some customers who received example job searches from their advisers interpreted these as lists of vacancies that they should apply for. In one case, this led to particular confusion and disappointment. Advisers should clearly communicate to customers that such listings are shared only as examples of job-search outputs, rather than suggested lists of vacancies to apply for.

Exit from service

When a customer leaves the service, they complete an exit review with their adviser. During this appointment, the adviser and customer repeat the Workstar assessment. This exercise gives advisers and customers the opportunity to reflect on achievements the customer has made, and what they might have gained from the process. This can be especially useful when customers are not exiting into employment.

Advisers also attempt to contact customers who enter employment 13 weeks after they leave the service. This is to collect information about their employment status and find out if they are in continued need of support. This improves monitoring of the service outcomes. But, as we mentioned in Chapter 4, it is not possible to get in touch with a lot of customers at this stage.

Summary

Ultimately, the question of standardisation is a complex one. There are clearly common elements to many Support to Work customers’ journeys. But the strength of the service lies in individual advisers’ ability to adapt the basic skeleton of the service in a way that suits the individual customer.

As one adviser summarises:

This underlines that employment support services should focus on and invest in the interpersonal skills of employment advisers. This should encourage effective communication with, and support of, a wide range of customers.

The Evaluation Stakeholder Panel was interested in exploring staff experience. To investigate we held a focus group with 4 Support to Work advisers and the customer coordinator.

We also held an individual interview with the Support to Work Programme Lead to understand their perspective. This gave us broader contextual information about who works in the service and how it operates.

In this chapter we discuss the following from the perspective of staff experience:

• Sources of job satisfaction.
• The culture of learning and sharing.
• Shared frustrations and concerns.

Sources of job satisfaction

It was clear from the staff focus group that advisers derive job satisfaction from working with Support to Work customers. Advisers enjoy noticing improvements in customers’ confidence over the course of the service. They also feel they are empowering customers with skills to navigate the job market independently.

Advisers also appreciate working in a service that reaches customers who may not be able to access other services. Providing the service remotely, means that geography is no barrier to participation. Advisers also feel that the service particularly suits customers with social anxieties who would find face-to-face, group, or community support programmes too challenging. Supporting groups of people they know would otherwise be excluded from services is gratifying.

Another central driver of job satisfaction is the level of engagement and honesty that customers bring to the service. Advisers believe that the voluntary nature of the service is at least partly responsible for this quality of engagement. Support to Work customers can choose for themselves when is the right time for them to be seeking help with finding work. This means customers are psychologically ready to engage fully with the support available, leading to a more positive and productive experience for customers and staff alike. Where employment support is mandated as a condition of receiving other support, this is not always the case.

Advisers also praised the realistic targets and indicators used to manage service performance. Staff highlighted how achievable outcomes targets allow them to focus on the individual customer and their personal goals, rather than on achieving job outcomes at any cost. This stands in contrast to previous employment support roles:

This realism in targets also offers advisers a greater degree of freedom in how they work with individual customers. Advisers can spend time building a trusting working relationship with a customer. When inducting staff into the service, the Programme Lead emphasises the importance of building such rapport.

As we discuss in Chapters 5 and 6, personalisation is highly important to customer satisfaction and outcomes too.

Recommendation for future practice: Maintain the realistic key performance indicators used in service management, which are currently set to the end of 2020. Much adviser and customer satisfaction with the service is directly related to the time advisers can spend getting to know and understand the individual customer. This should be an important consideration when setting ongoing performance targets.

The culture of learning and sharing

The evaluation stakeholders were also keen to understand if advisers feel confident when dealing with challenging situations. They wanted to explore whether there had been any change in this since the first Support to Work evaluation took place in 2018.

The focus group made clear that within the adviser team there is a commitment to mutual reflection, learning and support. This occurs both informally and through dedicated team meetings.

Advisers confirmed that they feel confident dealing with situations that require a safeguarding response. The Programme Lead also identified that cases where he must speak directly to a customer to resolve a situation have decreased significantly over time. This may suggest a growth in adviser confidence and resilience.

In monthly supervisions, individual advisers work with the Programme Lead to identify unique training needs and interests. Advisers may attend training individually and are then able to share what they have learned with the rest of the team.

Recommendation for future practice: We consider the open culture of learning and development to be central to positive staff experience and retainment. Any future versions of the Support to Work service should nurture this culture, as it supports adviser capability and maximises possible benefits to customers.

Shared frustrations and concerns

One of the main frustrations within the staff team is the amount of time spent setting and managing customers’ expectations. As we explored in Chapter 1, many customers place a lot of hope on Support to Work, especially if they have struggled to find employment support elsewhere. Many potential customers project their desires onto the service when they see an advert displaying its general themes. And as discussed in Chapter 3, the information available to new customers could be more explicit about the boundaries of what advisers can and cannot do.

Thus, advisers and the customer coordinator find they must repeatedly explain to customers that what they are not currently set up to do.

From our research we are aware that there are certain hopes and expectations that a lot of disabled jobseekers share. We address the feasibility and risks of Support to Work responding to some of these desires in Chapter 9 on possible gaps in the Support to Work service.

A separate adviser concern is how well the service supports deaf customers and customers with a hearing impairment. We also discuss this in Chapter 9.

In our qualitative research, we asked customers and staff about whether they had any suggestions for possible improvements to Support to Work. As a team with lived experience of disability, this is also something that we have reflected on as researchers.

In this chapter we report the most common suggestions from customers and staff. We appraise these in the context of information we’ve gathered about the circumstances and past experiences of customers using Support to Work.

We recommend that any plans to implement these suggestions are based on further research focusing specifically on relative benefits and risks.

The core suggestions fit into the following main categories and subheadings:

Extending the service offer

• In work support.
• Additional forms of peer support.
• Advice on identifying vacancies with disability-friendly employers.

Refining the current service offer

• Making appointment notes available in a written format.
• Assessing the support given to deaf customers and customers with a hearing impairment.

Extending the service offer

In work support

The past experiences of evaluation participants discussed in Chapter 1 are a reminder that disabled people face barriers not just in getting into work, but staying in work. Analysis by the Department for Work and Pensions shows that disabled people are twice as likely as non-disabled people to fall out of work.

Disabled people entering work may encounter particular challenges related to new employment. For example, they may have difficulty getting their employer to implement reasonable adjustments that enable them to conduct their role. There may be teething problems with Access to Work grants. And if someone has only recently acquired their impairment or condition, they may also be getting used to new ways of working.

People may also find that their benefits are unexpectedly affected, as happened to one customer we spoke to when they started their new job:

Customers explained that it would be helpful to continue to access Support to Work’s help and advice even once they have secured work. They said that the option of this support would provide them with something to ‘fall back on’ if they experienced difficulties with their transition into employment.

One customer we spoke to had been initially successful in getting a job after using Support to Work, but soon left again due to health issues. Situations like this could potentially have a better outcome if Support to Work customers who enter work could receive further support on schemes such as Access to Work and Fit for Work which aim to help disabled people stay in employment.

Customers taking part in the 2018 evaluation of the service also reported a desire to stay in contact with their adviser once in work. As a result of this 2018 finding and its repetition in the 2019 data, the Support to Work funders and managers have planned the introduction of 2 In Work Advisers to the service.

Identified action: From late 2020, Support to Work will start offering in work support to customers who enter employment during or shortly after using the service. 2 In Work Employment Advisers will provide this additional support through telephone and digital channels. Advisers will be able to help with aspects of the transition to work that commonly cause problems, such as the implementation of reasonable adjustments, dealing with the Access to Work process or ensuring benefits are not interrupted. Ultimately, it is hoped that this additional layer of support will lead to better rates of sustained employment for disabled people. The Evaluation team will need to monitor this by collecting data at agreed follow-up points to establish whether customers are still in work.

Additional forms of peer support

Both customers and staff suggested that Support to Work could be strengthened by connecting customers with each other directly. This would allow customers to learn from others in a similar position or from people who have previously experienced the same sorts of challenges. Some of the evaluation participants found that simply being in a focus group with others who had used the service was beneficial.

Ideas included networking events, direct phone calls between past and current customers, and an online forum connecting Support to Work customers. Scope’s online community already exists as a space where disabled people can share experiences, but participants’ comments suggest that awareness of it is low among Support to Work customers.

Recommendations for future practice:

• Ensure advisers proactively signpost all customers to Scope’s online community. Even if a customer has not expressed particular interest in accessing peer support.
• Consider creating a specific area of the online community for customers of Scope’s employment services, to encourage peer support between people with similar experiences.
• Explore the possibility of introducing additional formats of peer support between current and past customers. This could include networking events and telephone contact, weighing the potential benefits against safeguarding and data sharing risks.

Advice on identifying vacancies with inclusive employers

Given the negative experiences that many disabled people have had with previous employers, it is understandable that a lot of Support to Work customers are interested in identifying employers with positive attitudes towards employing and supporting disabled people.

Customers do not always trust in the government scheme, Disability Confident, that uses 3 levels of commitment to describe employers’ pledges to be inclusive. The scheme cannot convey to someone whether the individual line manager of a role or a given department or team will be inclusive and proactive in supporting disabled workers.

Accordingly, a lot of customers look to Scope and Support to Work for guidance on how to access inclusive employers.

While we heard some positive feedback that advisers help customers adopt alternative ways to ‘test’ a prospective employer’s attitude towards disability, some customers hoped that Support to Work could go further in helping to identify inclusive employers.

Identified Action: The 2018 evaluation of Support to Work also highlighted a demand for identifying inclusive employers. As a result, there are now plans underway to introduce a jobs board element to the service. This will allow customers to search for jobs from employers who have made a commitment to inclusive practices. The board will only be accessible to Scope customers and will not be visible to external search engines.

An Employer Engagement Specialist will be responsible for liaising with employers who are interested in posting vacancies to the board. These employers must commit to a partnership agreement that reflects the values and actions that Scope expects from inclusive employers.

The co-evaluation team understand the potential value of helping customers identify inclusive employers. This is a highly understandable desire, given the damaging previous experiences that customers have faced in the workplace. However, it is for the same reason that if not carefully managed, the system could pose risks to individual customers. If a customer encounters unhelpful attitudes and barriers when interacting with an employer on the jobs board, the gap between hope and reality will be even greater. This could potentially deepen disappointment and reinforce an individual’s sense of rejection and hopelessness.

We have urged the development of a clear system of accountability between Support to Work and employers advertising via the jobs board. If Support to Work customers report poor experiences with such employers, there should be a clear process for addressing this directly with the employer to ensure that it does not happen again. This process should also ensure that the customer is supported beyond the experience to prevent customers losing faith in Scope’s support.

The Programme Lead has agreed that there needs to be a clear process allowing customers to report their experiences with employers who advertise through the jobs board. This will be provided through the In Work Employment Advisers discussed in the previous section.

Future promotion of Support to Work will maintain a focus on equipping disabled people with the skills and confidence to become self-assured independent jobseekers. How the service is promoted links to existing challenges identified in Chapters 6 and 8 surrounding customers’ expectations of what Support to Work entails. Any promotion of the jobs board should be secondary to an emphasis on how the service develops individual skills and confidence. Marketing should also provide clear information that the service will expect active engagement from the customer to independently find and apply for jobs. This will be reflected in training and supervision of existing and new members of staff.

Refining the current service offer

We also received smaller pieces of feedback about the existing service which would increase its accessibility for some customers.

Making appointment notes available in a written format

One evaluation participant explained that it would be useful to receive a written summary of what they discussed with their adviser in each appointment:

This was echoed by a customer providing feedback on Support to Work via a feedback chatbot managed by Scope’s Insight team.

We appreciate that compiling appointment notes could result in significant extra work for advisers. But, it could also significantly benefit customers with certain impairments or conditions.

Recommendation for future practice: Encourage advisers to use their discretion in assessing, on an individual basis, which customers may significantly benefit from written appointment summaries. Ensure that customers are aware that receiving appointments notes is possible if it would significantly increase how accessible the service is to them.

Assessing the support given to deaf customers and customers with a hearing impairment

As we mentioned in Chapter 8, advisers have some concerns about how well the service currently supports deaf customers and customers with a hearing impairment.

Several options exist for customers with a hearing impairment when accessing the service:

1. Customers with access to assistive technology (such as a hearing aid and Roger Pen) can take part in telephone appointments.
2. Customers can choose to access the service exclusively through written communication by email, Skype chat and the portal messaging system.
3. Customers can have appointments where both their employment adviser and a British Sign Language (BSL) interpreter are present. The adviser sources the interpreter through a service called InterpretersLive!.

During the staff focus group, advisers expressed concern that customers in categories 2 and 3 do not receive the same depth and quality of support as those in category one. Advisers admitted a lack of confidence in using and promoting the InterpretersLive! service. This is reflected in service records showing that only one InterpretersLive! call was completed in 2019.

Advisers also suggested that establishing rapport with customers is much harder when relying solely on written communication.

Recommendation for future practice: We suggest introducing extra service steps when advisers identify that a customer has a hearing impairment. These could include:

• Explaining that the service is likely to run longer than 12 weeks if communication is taking place via written messages only. This would have an effect on service performance indicators and so would require further research to understand an appropriate adjusted timeline.
• Explaining that appointments with both an adviser and a BSL interpreter may take longer than typical phone appointments.
• Increasing awareness of, and signposting customers to, specialist deaf employment support services.

Identified Action: The Programme Lead has already given a refreshed demonstration of InterpretersLive! to the adviser team, led by advisers who are more confident using it. The adviser team will share experiences and methods for supporting customers with a hearing impairment on an ongoing basis during team meetings.

Our evaluation methodology included both qualitative and quantitative research methods. In this Appendix, we describe our sources, data collection methods and analysis processes.

The evaluation questions which guided our data collection were agreed by the Evaluation Stakeholder Panel. The questions are listed at the end of this Appendix.

Sources and data collection methods

Individual interviews with Support to Work customers

We held 14 individual telephone interviews with current or past Support to Work customers. The Evaluation Officer conducted 7 of these interviews. The remaining 7 were conducted by the 2 Co-Evaluators with the Evaluation Officer present.

We invited all customers who had accessed the service between 1 May 2019 and 25 November 2019 to take part in the evaluation by sharing their experience of the service. A copy of the email invitation we used is available on request. Interview participants were offered a £20 thank you voucher for their participation.

This sampling method means that our final group of participants was made up of a group of self-selecting individuals. The participants had decided for themselves that they were happy to provide an account of their use of Support to Work.

This is likely to have influenced the types of experiences we heard about during the interviews. Asking individuals to put themselves forward may particularly encourage participation from people who:

• had a particularly positive experience and would like to share this
• had a particularly negative experience and would like to make this known
• have time to take part in a research interview during office hours

We encourage stakeholders and readers to consider the self-selecting nature of this final qualitative sample when digesting the report’s findings. The sample is unlikely to be fully representative of the wider customer base. However, the themes and suggestions contained in the report reflect experiences that were shared repeatedly and independently across the sample. We include quantification of the number of participants experiencing certain themes where appropriate.

The interview questions were developed by the Co-Evaluators, Peer Reviewers, Customer Representative and Evaluation Officer. The questions drew upon the evaluation questions set by the Evaluation Stakeholder Panel.

A focus group with 5 further Support to Work customers

We held a face-to-face focus group with 5 current and past Support to Work customers at the Scope London office in Stratford. This ran for 2 hours with a break in the middle. Ruth, one of the Co-Evaluators, led the focus group and facilitated customers’ participation. Amy, the second Co-Evaluator, supported by taking contextual notes and asking follow-up questions. Jess, Evaluation Officer, was also present.

The invitation email used for individual interviews was the same one we used for recruitment to the focus group. People could choose to participate via either method. We paid for participants’ travel expenses and directly booked train travel where necessary. Focus group participants were offered a £50 thank you voucher.

This sampling method has the same limitations as the individual interviews. Additionally, holding the focus group in London placed a practical barrier to some customers being able to attend. We hope that by offering both remote and in-person participation options, we enabled any customer who wanted to participate the opportunity to do so.

Participants in the focus group remarked that it was useful to meet others in a similar situation and learn from their experiences. During the breaks, there was also a lot of conversation between the participants and Co-Evaluators. There appeared to be an open atmosphere, with participants building on each other’s comments throughout the session.

The focus group questions were developed by the Co-Evaluators, Peer Reviewers, Customer Representative and Evaluation Officer.

We have summarised the broad experiences of the interview and focus group participants we spoke to in Table A.1. The table highlights that 11 out of 19 evaluation participants had a clear positive experience with Support to Work. 4 had a clear negative experience, and 4 reported a mixed or ambivalent response to the service.

7 out of 19 participants were in work at the time of data collection, while 12 were not in employment.

Table A.1: Summary of evaluation participants, the broad nature of their service experience and whether they were in work at the time of the evaluation research

A focus group with 5 Support to Work staff members

To understand the staff perspective on the service, we held a 90-minute in-person focus group with 5 staff members working in Support to Work. The group did not include all staff members who worked in the service in 2019. Some team members were not available on the day of the focus group, and others were no longer working in the service.

The focus group questions were derived from the evaluation questions and from reflections by the co-evaluation team after speaking to Support to Work customers.

Amy led this focus group, with Ruth and Jess supporting with follow-up questions. The focus group took place during Support to Work staff members’ working hours and at their normal place of work.

An interview with the Support to Work Programme Lead

The Co-Evaluators co-led a Skype interview with the Support to Work Programme Lead after collecting qualitative data from customers and the adviser team. The co-evaluation team decided to schedule this part way through the project to add further context and a different perspective to some of the responses from customers and staff. As a result, the questions mostly related to emerging themes from the previous qualitative data collection.

Statistical analysis of before and after service outcomes surveys

All customers who start Support to Work are asked to complete a baseline evaluation survey online before starting the service. The survey measures customers in several different domains, including:

• how long the customer has been looking for a job
• how confident the customer is using the skills necessary for finding and applying for jobs
• whether the jobs the customer is applying for match their true interests and ambitions
• the customer’s sense of their own capability in work (for example confidence to manage their workload or talk to a manager about reasonable adjustments)
• the customer’s knowledge of employment rights

Customers then receive an email with the same survey when they leave the service, regardless of whether this is an early exit, an exit into work or when the service expires after 12 weeks. This is called the endline survey.

In theory, the combination of these surveys allows Scope to observe any changes that occur in these measures during the time that a customer is accessing Support to Work.

In 2019, only 24 customers completed both a baseline and an endline survey. We call these matched pairs. In total, 199 customers completed a baseline survey. 43 completed an endline survey.

Scope’s Data Analysis Manager and Evaluation and Impact Statistician analysed the data available from these 2019 baseline and endline surveys to explore the changes associated with using Support to Work.

Analyses were performed on the matched pair data and the baseline and endline survey respondents as groups.

The data presented in Chapter 4 mainly uses data from the matched survey pairs. A Wilcoxon Signed Rank test found that, excluding the measures on knowledge of employment rights, the overall changes displayed in the matched baseline and endline survey pairs in 2019 were significant (p < 0.05).

A one-way ANOVA test with Welch correction applied to the grouped baseline (n=199) and endline (n=43) surveys also found a significant difference between these groups (p < 0.05). Again, this is excluding the measure for knowledge of employment rights.

There was no significant change in the number of customers answering questions on knowledge of employment rights correctly before and after the service. We have recently updated these questions in the outcomes survey as we suspected that the original questions may not have been sensitive enough.

The new questions cover reasonable adjustments, disability discrimination and information sharing around disability.

Statistical analysis of delivery data stored in the service database

Information about customers and how they use Support to Work is stored in Microsoft Dynamics, a Customer Records Management system. The database holds record of a customer’s interaction with the service, including:

• emails between advisers and customers
• appointments booked, attended and cancelled
• action plans
• relevant milestones, for example when a customer started on the programme
• status (describing whether someone is an active, expired or exited customer, or if they have entered employment)

It also stores demographic data about the customer base.

Scope’s Data Analysis Manager and Evaluation and Impact Statistician extracted and analysed data from the Dynamics database to answer some of the evaluation questions detailed at the end of this Appendix. They used the data to explore things such as the average number of appointments customers have before they enter into work or before customers leave the service.

Thematic analysis of free text data stored in the service database

The service database also contains several free text fields which advisers use to record context about a customer’s interaction with Support to Work. These case note fields can contain background information about a customer, details of what advisers talk to customers about, and reasons for customers leaving the service early.

We used this free text data to explore the main reasons for customers leaving the service early (see Chapter 3). To isolate the relevant sections of free text, The Evaluation Officer searched the exported case notes for the terms ‘exit’, ‘leave’, ‘no longer’ and ‘end’. One of the Peer Reviewers then thematically sorted the selected notes into categories and derived the main reasons for early service exits detailed in Chapter 3.

Statistical analysis of delivery data stored in Call Handling

Call Handling is the virtual call centre software that Support to Work uses for running and managing its telephone calls with customers. It generates and records data about all calls that take place within the service. It captures information such as:

• which adviser was on a call
• how long the call was
• when the call took place
• whether a call successfully connected
• the number of calls to a particular number

Scope’s Data Analysis Manager and Evaluation and Impact Statistician used call handling data to explore the relationship between aspects of service provision (such as the number of calls or the amount of time spent in calls) and positive outcomes.

Statistical analysis of the Workstar Outcomes Tool

The ‘Workstar’ is a tracking tool developed by Triangle. Advisers and customers complete it together at the start and the end of a customer’s time with Support to Work. Advisers ask customers to rate themselves in 7 areas that are relevant to success in finding work. The scale runs from 1 to 10 and the 7 items are:

• job-specific skills
• aspiration and motivation
• job-search skills
• stability
• basic skills
• social skills for work
• challenges

The Workstar is used both as a diagnostic and outcomes measurement tool. Advisers use it to better understand a customer’s situation and what they may need most from Support to Work. At the same time, by completing it at the start and the end of the service, customers can track changes in how they feel about these 7 important areas and any progress achieved during their time on the service.

Scope’s Data Analysis Manager explored the relationship between Workstar ratings and other outcomes measurements such as the outcomes survey and actual employment.

Qualitative analysis

To analyse the qualitative data, the Co-Evaluators and Evaluation Officer developed a coding framework derived from the evaluation questions. We applied this coding framework to the interview and focus group transcripts.

The Evaluation Officer coded all 17 transcripts (customer interviews, customer focus group, staff focus group, and Programme Lead interview). Together the Co-Evaluators coded 7 transcripts. In addition to this systematic analysis, the Peer Reviewers also read and shared reflections on 7 transcripts. The Co-Evaluators read and became familiar with the whole data set.

After the first round of coding, we compared our individual code applications and reflections in team meetings. We further updated the coding framework and re-read transcripts to ensure that all relevant data was captured and coded. The Evaluation Officer than imported all coded excerpts into a master document arranged by evaluation question. This formed the basis for further thematic analysis of the collected data, which we discussed continually as a team.

This master document underpins the data presented in the final report and is the source of the supporting quotations used as evidence throughout our chapters.

Further information on how we have worked together as a co-evaluation team will be available in our separate Guide to Co-Produced Research and Evaluation that we aim to publish at a later date.

Table A.2: Condensed Evaluation Questions for Support to Work Evaluation 2019

Back to the top

As authors of this report, we would like to express our thanks to several people and organisations who helped make this co-produced evaluation possible.

Firstly, thank you to Peer Reviewers Nathan Giles and Julie Hartley, who provided thoughtful and constructive commentary to the full evaluation process, from design through to writing the report. Your contributions were invaluable.

We are grateful to Mushtaq Patas, the project’s Customer Representative, who helped us ensure that our research tools were relevant and sensitive to the experiences of Support to Work customers.

Our great thanks to all of the individuals who took part in our research, either as a Support to Work customer or member of staff. We deeply appreciate the time and thought you gave to your participation.

We extend many thanks to Virgin Media, who made a vital contribution to both Support to Work and its evaluation, as funder and stakeholder. Support to Work was launched in 2018 as part of Scope’s partnership with Virgin Media. Our partnership ambition is to support 1 million disabled people with the skills and confidence, to get into and stay in work by the end of 2020.

We express particular gratitude to Michele Scattergood of Breakthrough UK, who brought the perspective of a Disabled People’s Organisation to the evaluation. Michele contributed crucial suggestions on research priorities and appropriate ways to disseminate research findings.

The quantitative analysis within the report would not have been possible without the work of Giles Skerry, Scope’s Data Analysis Manager, and Renee Davis, Scope’s Evaluation and Impact Statistician. The report’s final form would not have been reached without thoughtful review and engagement from Guy Chaudoir, Anne Curran, Susannah Douglas, Mary McFadden and Cara Pears.

Thank you too to Graham Findlay, Co-Production Lead, who shared enormous wisdom in the setting up of this project.

Finally, thank you to all the stakeholders on this project’s Stakeholder Panel, who helped define how we conducted and shared this research. We are delighted to share it with you.

Amy Frounks, Ruth Murran and Jessica Bricknell

Back to the top