On 3 December 2019, the results of the 2018 cycle of the Programme for International Student Assessment (PISA) will be announced. There is sure to be significant interest as to whether Wales improves on its results in previous cycles, which have generally been regarded as disappointing.
This article explains what PISA is, how Wales has fared to date and its relevance to the education policies being pursued by the Welsh Government.
What is PISA and what does it measure?
PISA is a survey undertaken every three years, which evaluates education systems worldwide by testing the skills and knowledge of a sample of 15-year-olds in each participating country. It is run by the Organisation for Economic Co-operation and Development (OECD).
The tests aim to measure something different to traditional examinations. They focus more on the ability to apply knowledge and skills to real life challenges whereas GCSEs, for example, have historically tended to test pupils’ knowledge of the curriculum.
PISA reflects the changing nature of the economy and the labour market. For example, the OECD has said (PDF):
The goal of education has continued to shift its emphasis from the collection and memorisation of information only to the inclusion of a broader concept of knowledge. (…)
The ability to locate, access, understand and reflect on all kinds of information is essential if individuals are to be able to participate fully in our knowledge-based society.
New versions of GCSEs introduced in Wales since September 2015 have sought to adopt more of a skills-based approach and the new Curriculum for Wales aims to take this further by prioritising skills alongside knowledge and experience.
PISA focuses on knowledge and skills towards the end of compulsory schooling. In most countries the end of compulsory education is around the age of 15, when, as the OECD comments, students are supposed to have mastered the basic skills and knowledge to continue on to further/higher education, training or employment.
The OECD further describes (PDF) the aims of PISA as follows:
- a system-level assessment, representing a commitment by governments to monitor the outcomes of education systems;
- policy-oriented by linking data on students’ learning outcomes with data on key factors that shape learning in and out of school;
- carried out regularly to enable countries to monitor their progress in meeting key learning objectives;
- assesses both subject matter content knowledge, on the one hand, and the capacity of individuals to apply that knowledge creatively, including in unfamiliar contexts, on the other.
Examples of PISA questions can be found here on the OECD website.
Which tests are taken and how is the sample of pupils selected?
There are three main domains to PISA: Reading, Mathematics and Science. These alternate so that one of these is the lead domain in each three-yearly cycle. As the lead domain in PISA 2018, there were more questions to assess reading skills than the other two domains and there were will be additional analysis of the results.
The PISA tests were taken by a sample of 15-year-olds in Wales between late October and mid-December 2018. The sampling details will be provided in the report on Wales’ results by the National Foundation for Educational Research (NFER) on 3 December 2019. To give some context, 3,451 pupils and 140 schools participated in PISA 2015 and there are around 30,000 students in Year 11 and 187 secondary schools currently in Wales.
The Welsh Government provides a list of all eligible schools and populations to the PISA consortium which in turn selects an ‘original main sample’ to take part. Schools can then decline to participate, in which case schools from replacement samples may be used. A sufficient number of the original selected sample must participate in order to satisfy PISA requirements.
Variables such as school type, language-medium, gender, local authority and region are used to ensure a fair and representative sample. The identity of schools and pupils taking part is not disclosed.
What have Wales’ previous results been?
Wales took part in PISA for the first time in 2006. As shown by the infographic below:
- Wales’ scores in the most recent cycle (2015) were lower in each domain than 2006.
- Wales’s scores in PISA 2015 were also lower than each of the other three UK nations, as well as the OECD average, for each of the three domains.
PISA 2015: Mean scores across the UK
Source: UCL, Achievement of 15 Year Olds in Wales: PISA 2015 National report (2016); OECD, UK country note (2016) (PDF)
When the 2015 results were published in December 2016, the Minister for Education, Kirsty Williams AM, acknowledged that ‘we are not where we want to be’ but paraphrased the ‘unambiguous’ advice of the OECD (PDF 2.91MB) as ‘stay the course; be brave; you are doing the right things’. The action plan, Education in Wales: Our National Mission 2017-2021 sets out the Welsh Government’s plans to improve educational standards.
What is the Welsh Government’s target for PISA?
The Welsh Government has a target that Wales achieves 500 points in each of three domains (Reading, Mathematics and Science) by PISA 2021. This target was set in 2014 after a previous target (set in 2011) that Wales be in the top 20 ranked PISA countries by PISA 2015 was revised.
The Welsh Government confirmed the 500 point target in summer 2017 after the Minister for Education had initially said ‘it was not her target’, albeit placing it in the context of other challenges such as stretching potential high-achievers. In February 2019, First Minister, Mark Drakeford AM, reiterated that the target is to achieve 500 points in each domain by PISA 2021.
How has PISA influenced Welsh Government policy?
The OECD and PISA have significantly influenced Welsh Government policy towards school improvement over the past decade.
The publication of the PISA 2009 results in December 2010 delivered what the then Minister for Education, Leighton Andrews AM, called a ‘wake up call to a complacent system’ and ‘evidence of a systemic failure’. Wales’ disappointing PISA results precipitated a renewed and changed focus on school accountability, a return to the basics of literacy and numeracy and a new regional approach to school improvement, all set out in the then Minister’s twenty point plan. Between 2013 and 2016, Huw Lewis’ time as Minister continued this approach and focused on tackling the link between deprivation and low attainment. Over time, there has also been a recognition that Wales needs to improve support for more able and talented learners if a greater number are to achieve the highest grades.
The OECD has been part of the drive to find solutions to school improvement. Its reports in 2014 (PDF 3.75MB) and 2017 (PDF 2.91MB) informed the Welsh Government’s education action plans, Qualified for Life and Education in Wales: Our National Mission 2017-2021 respectively.
The PISA 2018 results, when published on 3 December 2019, should give a clearer picture of Wales’ progress in raising standards in schools over the past decade and whether the Welsh Government is on track to meet its target of 500 points by 2021.
Article by Michael Dauncey, Senedd Research, National Assembly for Wales
