Wales’ scores in the Programme for International Student Assessment (PISA) have fallen further behind the rest of the UK and the international average, as shown by results published earlier today.
PISA evaluates countries’ education systems based on the performance of a sample of 15 year-olds in standard tests in Mathematics, Reading and Science.
Globally, much attention is focused on countries' top-line PISA scores and rankings, but the programme also provides a range of information and analysis. Overall PISA’s purpose is to help governments improve their young people's education.
Delayed a year from 2021, the 2022 cycle was the first to be held since the emergence of COVID 19 and is therefore set against the considerable impact of the pandemic on schools. The Minister for Education and Welsh Language points to the pandemic and the disproportionate impact on areas with higher levels of deprivation, as among the reasons for the decline.
In summary, the results show:
- Wales’ scores have decreased since 2018 in Mathematics, Reading and Science by 21, 17 and 15 points respectively.
- The other UK nations’ scores have also decreased in each of the three subject domains. However, the level of decrease is greater in Wales than all other UK nations in all three domains.
- After improving relative to the Organisation for Economic Co-operation and Development (OECD) average in 2018, such that there was no statistically significant difference, Wales’ scores are again significantly lower than the OECD average in all three domains.
- Wales’ scores are the lowest they have ever been since first participating in PISA in 2006.
The Minister says that, whilst the pandemic was expected to cause a decline in performance, “that does not make these results any less disappointing”. He says the pandemic has “derailed” previous improvement and it “requires a national effort to turn thing around”.
The official report on Wales’ results, produced by the University of Oxford, provides further information and analysis on a range of aspects of PISA.
Table 1: Wales’ mean scores in PISA over time
|
|
2006 |
2009 |
2012 |
2015 |
2018 |
2022 |
|
Mathematics |
484 |
472 |
468 |
478 |
487 |
466 |
|
Reading |
481 |
476 |
480 |
477 |
483 |
466 |
|
Science |
505 |
496 |
491 |
485 |
488 |
473 |
Source: Welsh Government, PISA national reports (several editions)
Table 2: UK nations’ mean scores in PISA 2022
|
|
Mathematics |
Reading |
Science |
|
Wales |
466 |
466 |
473 |
|
England |
492 |
496 |
503 |
|
Scotland |
471 |
493 |
483 |
|
Northern Ireland |
475 |
485 |
488 |
|
OECD |
472 |
476 |
485 |
Source: Welsh Government, PISA national reports (several editions)
Note: The OECD advises caution in comparisons between different countries, and particularly so in 2022 where a number of countries, including England and Scotland, are estimated by the OECD to have a possible upward response bias of 7-9 points in maths and reading.
Wales’ PISA scores
Source: Welsh Government, PISA national reports (several editions)
What is PISA and what is its influence?
PISA is organised by the Organisation for Economic Co-operation and Development (OECD) and usually held every three years. Wales has participated since 2006. Around 80 countries took part in 2022.
PISA aims to test the extent to which 15 year-olds have “acquired the knowledge and skills that are essential for full participation in modern societies”, i.e. what it is important for young people to know and be able to do. The OECD explains:
The assessment does not just ascertain whether students can reproduce knowledge; it also examines how well students can extrapolate from what they have learned and can apply that knowledge in unfamiliar settings, both in and outside of school. This approach reflects the fact that modern economies reward individuals not for what they know, but for what they can do with what they know.
PISA and the OECD have had a considerable influence on Welsh education policy for over a decade since the 2009 results delivered a “wake up call to a complacent system” and "evidence of systemic failure”. Previous Welsh Governments asked the OECD to help steer their approach to school improvement, with OECD reports in 2014 and 2017 informing previous education action plans.
How many pupils took the tests and how were they selected?
2,568 pupils aged 15 in 89 schools across Wales took the PISA 2022 tests in November and December 2022, during a two hour computer-based exercise. For context, there are around 33,000 pupils in Year 11 in 178 secondary schools and 27 middle schools in Wales. Participating pupils and head teachers also completed a questionnaire, the results of which are summarised in chapter 7 of the University of Oxford’s report.
The majority of the pupils taking the tests were in Year 11, while some were in Year 10. All were aged 15. The report gives further information on sampling arrangements (see section 1.3.4 and Appendix A).
Does the Welsh Government have a target for PISA?
It did, but not any more.
In 2011, as the Welsh Government was responding to the shock of the 2009 results (which were announced at the end of 2010), it set a target for Wales to be in the top 20 PISA nations by 2015. It changed this in 2014 to a target more within its own hands - achieving 500 points in each of the three domains (Mathematics, Reading and Science) by 2021. As stated earlier, the 2021 cycle was put back to 2022 because of the pandemic.
The First Minister confirmed in 2019 that the 500 point target remained in place. The 2021 assessments were delayed to 2022 because of the pandemic and the Minister for Education and Welsh Language revealed in November 2022 that the Welsh Government no longer held the target. Instead, he said it “does not have a numerical target” but wants learners to “give a strong account of their capability” and for Wales to learn from taking part.
We can now see that Wales would have missed those targets. Wales’ scores were 34 points short of 500 in Mathematics and Reading, and 27 points short in Science. Of the other UK nations, only England reached 500 points in any of the domains, scoring 503 in Science.
But PISA isn’t the only measure
Other measures of educational performance, such as GCSEs, A Levels, vocational qualifications and personalised assessments in reading and numeracy are arguably more important to the people undertaking them and the professionals teaching them. Indeed, some people have highlighted PISA’s shortcomings, including internationally and in Wales.
The results of those other measures, beyond PISA, show the challenges the Welsh Government’s “national mission” to improve educational standards faces. While changes to performance measures and the suspension of exams during the pandemic limit comparisons between years, we can analyse the relative attainment position between pupils. The GCSE attainment gap between deprived pupils and their peers widened between 2016 and 2022. The 2023 GCSE data is due to be published on 7 December. We wrote about the initial data available on the 2023 GCSE and A level results when they were released in August.
Data published last month on the personalised assessments taken by pupils in Years 2 to 9 in Reading and Numeracy also shows a downward trend, coinciding with the pandemic. In 2022/23, pupils had declined by an average of 11 months in reading Welsh and four months in reading English, compared to pupils of the same age in 2020/21. The decline was four months on average in numeracy (procedural) compared to 2018/19, while attainment was the equivalent of six months higher in numeracy (reasoning) than in 2021/22.
Another important measure of standards is provided by the education inspectorate, Estyn, which inspects and reports on the quality of provision and outcomes in schools and other settings. Estyn’s Chief Inspector published his summary findings from 2022/23 in October and will publish the annual report in the New Year. Here’s our article about last year’s report.
What is the Welsh Government doing to improve educational standards?
We wrote in September about the Welsh Government’s Programme for Government objective to “continue our long-term programme of education reform, and ensure educational inequalities narrow and standards rise”. We also wrote last week about the importance of the education workforce to school improvement and various "teaching matters", including recruitment and retention issues.
The Minister for Education and Welsh Language, Jeremy Miles, has committed to prioritise “high standards and aspirations for all”. He published a new school improvement framework in summer 2022, and a ‘roadmap’ in March 2023 giving an updated overview of the Welsh Government’s education priorities.
Responding to the discouraging data on personalised assessments (and possibly anticipating today’s PISA results), the Minister announced last week a new Mathematics and Numeracy Plan, and an updated oracy and reading toolkit.
Following the written statement he has already issued, the Minister will make a statement in the Senedd this afternoon regarding the PISA results. You can watch this live or retrospectively on SeneddTV and a transcript will be available shortly afterwards.
Article by Michael Dauncey, Senedd Research, Welsh Parliament
