Evaluation of CQC's local authority pilot assessments

The pilots have provided a thorough opportunity to test CQC’s approach to assessing local authorities. Despite early trepidation from the local authorities, intensive timescales and the adjustment to a whole new assessment approach, it has been largely well-received. They also reported that it was largely proportionate to their expectations on reflection, acknowledging this would always be a big piece of work.

The assessment team has also taken on the challenge of simultaneously managing 5 different pilots while also adjusting to a new assessment framework. It is clear that this has been task on a huge scale for those involved. It has confirmed that elements at the heart of CQC’s approach, such as the quality statements, are fit for purpose and CQC has created a methodology that can be applied in such a way that accommodates a range of contextual factors about different local authorities.

The most prominent and recurring themes in the feedback have been around how more time and more guidance would have alleviated many of the challenges with the processes that supported the assessment. Time has been an issue due to the pressures of the pilots, and it is unclear at this stage to what extent similar timeframes will be replicated in the initial formal assessment period. The consensus is similar to what we heard in the test and learn exercises – that CQC must do more ahead of the fieldwork. Having significantly longer lead-in times to use the information gathered ahead of fieldwork will enable the fieldwork to focus on the most pertinent opportunities to engage the right people, supplementing what CQC already knows.

Guidance and more specificity about what CQC wants to understand about local authorities has been heard throughout, from the initial information return process, the fieldwork planning and in the fieldwork itself. It is now time for CQC to move away from the relatively open approach taken so far to one that is much more defined. Local authority feedback indicates they would welcome this, and it should result in a more efficient approach all round.

The team would have also benefitted from more guidance on some key areas, such as the allocation of evidence and, quite simply, how much evidence is enough. When it came to scoring, the team aired genuine concerns about the lack of guidance on the model and would have valued guidelines around what constituted different ratings. Without this there is concern for inconsistency and challenge in judgements if teams have no formal guidance to fall back on. CQC will need to carefully consider how it supports these requests, also considering the new staff who will be joining the local authority assessment team and the importance of training and inducting them well to all aspects of the new approach.

The full value and role of assessment reports will be seen more once they are published, but we know improvements to their structure have been made. If these improvements were coupled with more guidance for the team on the level of evidence to include, which strikes the balance of succinctness without losing meaning, the team would have more confidence in the reports. As it stands, not everyone is clear on the primary audiences for reports so this needs to be conveyed as CQC starts the initial formal assessments.

From the local authorities’ perspectives, the pilot assessment reports helped them to reaffirm areas for improvement. Some local authorities felt they could go further in providing recommendations and clear steps to improve ratings. All felt there should be consideration in judgements for if or how local authorities were already addressing identified areas for improvement. Local authorities raised some concerns about the strength of corroborating evidence in relation to some judgements, and a lack of cohesion between the narrative and judgements at times. They all expressed wanting to understand the scoring model and process more, and earlier in the pilots.

As the pilots come to a close, the team and the local authorities are acutely aware that the formal assessment period is set to begin in earnest. There is apprehension about this period and whether there is sufficient scope for the learning from the pilots to bed in, while ensuring CQC takes a consistent and clear approach into the next phase. While the methodology fared well in the pilots, the ultimate test of this will come next when the true range of local authorities will be subject to this approach. This will include local authorities that may have challenges that test the methodology in a way that CQC did not encounter in the pilots, with what were a relatively willing audience. It will also be vital to the success of the approach that CQC is mindful of communicating frequently and thoroughly with the local authorities who are greatly interested in how they can best prepare themselves.