Committees are sometimes called the ‘kitchen’ of a parliament, where a significant amount of work is done, often unseen. But how do we know if they’re working in the best way? How can we measure their effectiveness when the nature and context of their work is so complex?
Last year Professor Diana Stirbu of London Metropolitan University was commissioned through the Senedd Research Academic Fellowship Scheme to explore the power, influence, and impact of Senedd committees. The aim was to develop a way to evaluate the effectiveness of committees in the Sixth Senedd.
Professor Stirbu’s final report is published today, which has been endorsed by the Senedd’s Chairs’ Forum.
What do committees currently do?
Senedd committees don’t currently have an agreed way to measure their effectiveness. Professor Stirbu’s research discovered existing good practices, like monitoring the implementation of committee recommendations, and doing follow-up scrutiny.
But these good practices often happen inconsistently and in isolation. The research concluded that evaluation should be integral to committees’ strategic approach, to ensure coherence between their objectives, their desired long-term impact/outcomes, and what they actually do. This includes goal-setting, planning future work, monitoring, tracking progress, and assessing the legacy of work.
To examine how this could be achieved, the research looked at:
- the conditions for effective committee activity, and
- the conditions for effective evaluation and self-reflection to take place.
How was the research done?
Professor Stirbu reviewed the current evidence and literature on effectiveness of parliamentary committees. This was followed by field research, involving interviews with politicians and officials, group discussions, and collaborative workshops. She also spoke to external and international stakeholders.
What do effective committees look like?
Professor Stirbu found a shared understanding of what effective committees look like. They:
- have Members who are fully engaged and interested in their work;
- are supported by excellent services, have access to external advice and expertise, and operate within structures where corporate goals are aligned with committee business;
- can strategically plan and manage their work, with clear goals and purpose. They focus on outcomes, not activities, and have a clear, shared idea of success and how it’s evidenced;
- foster partnership and joint approaches, are evidence-led, promote lesson learning, engage in evaluation, self-reflection and continuous improvement;
- work transparently and communicate their work effectively to a wide range of audiences. Their reports are user-friendly, and tell compelling stories about their work and its impact. They have good relationships with a wide and diverse range of stakeholders, which helps them to stay relevant; and
- are ambitious and creative in reaching out to new audiences, designing activities that are fit for purpose and maximise impact, and bring in lived experiences.
How can effective evaluation and self-reflection take place?
The research found ways for committee to improve their work and measure effectiveness.
These include: better data collection, addressing structural and cultural barriers to Members’ engagement in evaluation and self-reflection, more meaningful engagement with Government responses (beyond just acceptance), external input into evaluation, and regular feedback on committees’ work.
Getting more evidence from a wider range of people
Public engagement and diversity of evidence were cross-cutting themes in the research. It revealed that committees’ impact is seen to depend, to an extent, on how well they communicate with a wide range of stakeholders. But it also found these to be important standalone functions themselves.
It uncovered good practice and lessons from previous experiences. This included a genuine drive for meaningful engagement from Members and officials, diverse approaches to engagement used by committees. It also found further opportunities for committees to use digital engagement to collect a wider range of evidence.
The barriers committees face in collecting a wide range of evidence include differing interpretations of what ‘engagement’ means, and a lack of monitoring data about the diversity of their evidence. A ‘weak’ Welsh media was also cited as a challenge to public engagement and the communication of committee work.
What can be done differently in the Sixth Senedd?
The report recommends that evaluation of committees should be done at three levels:
- by individual committees;
- at Senedd corporate level, to ensure a wider perspective, and
- externally, for a longitudinal, independent perspective.
Professor Stirbu suggests a framework to help committees make sure there’s coherence between their strategic goals, their planned activities, and the process in place to regularly review their impact and influence.
This is presented as a ‘theory of change’ model, which is a description of why a particular way of working will be effective, showing how change happens in the short, medium and long term to achieve the intended impact.
It suggests that a measurement framework should start from what long term impact and outcomes committees want to achieve. It should then set out how the committee will know if it has delivered on its ambitions, from which a series of measures and ways of evidencing success (or progress) are developed.
What about other parliaments?
The UK House of Commons Liaison Committee examined the effectiveness of select committees in its 2019 report. Its recommendations covered a broad range of issues, including: strategic planning, accountability, evidence, engagement, research, communications, gender balance, and resources.
What happens now?
Professor Stirbu’s report has been endorsed by the Chairs’ Forum (which is an informal committee made up of all Senedd committee chairs). The recommendations will be implemented across the Senedd.
Work has already started, with the launch of committee evidence diversity monitoring this month, and the Senedd’s new knowledge exchange strategy.
You can read Professor Stirbu’s full research and recommendations in her report, ‘Power, Influence and Impact of Senedd Committees: Developing a framework for measuring committees’ effectiveness’.
Senedd committees play an important role in holding the Welsh Government to account and scrutinising draft laws. This research shows that there are opportunities to improve how they work, for the benefit of the people of Wales.
Article by Hannah Johnson, Senedd Research, Welsh Parliament
