Assessing quality and performance

We will use the best methods to collect evidence depending on the type of evidence needed for a specific quality statement.

We will continue to build on our existing methods for collecting evidence. Although we will assess the evidence using a new framework, most of the information we consider will be similar to what we have been looking at in the past.

We will also apply the same rules when giving notice of assessments. This includes where we carry out unannounced on-site activity.

We will email you to tell you when an assessment is starting and may ask you for some types of evidence at this point.

We will give feedback to the provider when we have completed either an on-site or off-site assessment. If possible, we will give feedback about on-site activity immediately after completing it.

Methods we will use to gather evidence

The evidence we use in our assessments of quality may be gathered through both on-site and off-site methods. On-site activity remains really important and we expect to use our time visiting services in a more targeted way.

On-site activity

We will spend our time on site:

• observing care and how staff interact with people
• observing the care environment, including equipment and premises
• talking with people using the service
• talking with staff and service leaders.

We will carry out site visits when it’s the best way to gather the evidence we need. For example, we’ll do this:

• where people have communication needs that make telephone or video conversations challenging, or not suitable
• where there are concerns around transparency and confidentiality, for example to make sure someone isn’t overheard or being influenced by others
• to check the validity of evidence we have already gathered in a setting

We may carry out a site visit to collect evidence without giving notice beforehand. We would do this, for example in response to a specific concern.

We will carry out on-site activity more frequently in settings where:

• there is a greater risk of a poor or closed culture going undetected in a service
• it is the best way to gather people's experience of care
• we have concerns about transparency and the availability of evidence
• we have a statutory obligation to do so, for example as a member of the National Preventative Mechanism we must visits places of detention regularly to prevent torture and other ill-treatment

Information we collect from national bodies

We will continue to use and develop insight from national data collections, particularly where there are nationally agreed measures of quality. For example:

• capacity tracker for adult social care services
• electronic staff record
• GP patient survey
• hospital episode statistics
• Learn from Patient Safety Events (LFPSE)
• measures from the National Clinical Audit Programme
• mental health services data set
• national SITREP information
• NHS staff survey
• prescribing datasets
• Skills for Care
• waiting times

We have a programme to manage the external data sources we use. Updates to the data sources depend on the schedules set by the bodies responsible for them.

Information we collect from providers

We will continue to:

• request an annual provider information request (PIR) from adult social care services, using the existing collection method
• carry out online reviews of clinical records
• request evidence directly from providers to support an assessment, most likely by email.

Where we request evidence, we will use information that a provider has available. Apart from the PIR for adult social care services, we will not specify a particular format for the information. We know services are at different levels of digital maturity so we will adapt our collection methods while they develop, in line with the plan for digital health and social care.

For now, providers do not need to submit evidence to us proactively. We will ask you for anything we need.

In addition to specific requests for evidence from providers, we will continue to carry out interviews with staff and workers in services, and with service leaders. We may do this online.

We will determine when we need to request evidence directly from providers based on the timetable for assessments.

Feedback we receive and our engagement activities

We will continue to use the feedback we receive from people and their representatives about their experiences. This could be:

• from our Give feedback on care service
• when people contact us, through our National Customer Services Centre

We will also continue to:

• run online focus groups or contact people with experience of using a service
• commission the NHS Patient Survey Programme to understand people’s experiences of care.

We will also work with other people and organisations to help us collect evidence, for example local Healthwatch groups and our Experts by Experience. They can help us reach out to people, families and carers, and engage with communities whose voices are seldom heard.

The timetable for collecting these sources of evidence depends on the specific source. For some sources, such as Give feedback on care, this is ongoing. Other evidence sources will be updated less frequently – such as through patient surveys (which are conducted annually or every 2 years).

How we will use evidence in our assessments

We will bring together all relevant evidence for a specific evidence category. We will then make a judgement based on the requirements of the quality statement and the new scoring scale. As we do this, we will consider:

• whether the evidence collected covers the scope of the service sufficiently
• the quality and validity of the evidence.

Specialist support

To inform how we collect and use evidence, our teams will engage the expertise of our:

• Experts by Experience
• specialist advisors
• executive reviewers (colleagues who support on assessments of the well-led key question for NHS trusts).

Assessment teams can get quick access to specialists to support them to:

• understand which evidence to collect
• corroborate and analyse evidence
• interview key staff.

This helps ensure that our judgements maintain credibility.

Deciding what evidence collection methods to use

The evidence we collect and how we collect it depends on a combination of factors. We will take into account:

• the type of service
• the quality statement and relevant evidence category
• the information we already hold about a service

There isn't a full list of evidence that fits every service. We may need to follow up specific risks or circumstances that would need particular evidence. We do not want providers to prepare specific documents – rather we ask for information they already have.