Assessing quality and performance
Example: how we reach a rating
To assess quality against a particular quality statement, operational colleagues will look at the relevant evidence categories. In this example, we are just looking at the 'infection prevention and control' quality statement.
Infection prevention and control: "We assess and manage the risk of infection. We detect and control the risk of it spreading and share any concerns with appropriate agencies promptly."
For this service, the key evidence categories for this quality statement are:
- People's experiences
- Feedback from staff and leaders
- Observation
- Processes
We would look at individual pieces of evidence under each evidence category and based on the strength of what we find, give a score of 1 to 4.
For example, in the ‘people's experience’ evidence category, we may look at:
- patient surveys
- complaints and compliments
To gather evidence in the ‘feedback from staff and leaders’ and ‘observation’ categories, we might schedule:
- an inspection to look at the care environment
- a call to speak with staff at the service.
We would then combine this new evidence with what we already hold on ‘processes’ to help us form a view of quality.
Example: combining evidence category scores to give a quality statement score
| Evidence category | Score | Existing or updated score |
|---|---|---|
| People's experiences | 3 | updated |
| Feedback from staff and leaders | 2 | updated |
| Observation | 3 | updated |
| Processes | 3 | existing |
| Total score for the combined evidence categories | 11 |
We calculate this as a percentage so that we have more detailed information at evidence category and quality statement level.
To calculate the percentage, we divide the total (in this case 11) by the maximum possible score. This maximum score is the number of relevant evidence categories multiplied by the highest score for each category, which is 4. In this case, the maximum score is 16. Here, it gives a percentage score for the quality statement of 69% (this is 11 divided by 16).
We convert this back to a score. This makes it easier to understand and combine with other quality statement scores to calculate the related key question score.
We use these thresholds to convert percentages to scores:
- 25 to 38% = 1
- 39 to 62% = 2
- 63 to 87% = 3
- over 87% = 4
In this case, the percentage score of 69% converts to a score of 3.
We then use this score to give us an updated view of quality at key question level. In this case it is for the safe key question:
Example: combining quality statement scores to give a key question rating
| Quality statement | Score | Existing or updated score |
|---|---|---|
| Learning culture | 2 | existing |
| Safe systems, pathways and transitions | 3 | existing |
| Safeguarding | 3 | existing |
| Involving people to manage risks | 2 | existing |
| Safe environments | 3 | existing |
| Infection prevention and control | 3 | updated |
| Safe and effective staffing | 2 | existing |
| Medicines optimisation | 3 | existing |
| Total score for the safe key question | 21 |
Again, we calculate a percentage score. We divide the total (in this case 21) by the maximum possible score. For the safe key question, this is 8 quality statements multiplied by the highest score for each statement, which is 4. So the maximum score is 32. Here, it gives a percentage score for the key question of 65.6% (this is 21 divided by 32).
At key question level we translate this percentage into a rating rather than a score, using these thresholds:
- 25 to 38% = inadequate
- 39 to 62% = requires improvement
- 63 to 87% = good
- over 87% = outstanding
Therefore, the rating for the safe key question in this case is good.