Schools across Wales have come to us with misunderstandings about this recent WG announcement. The page on Learning Wales is entitled “Reporting on the National Literacy and Numeracy Framework is changing” – this is important because the announcement is about the reporting of the Literacy and Numeracy Framework specifically.

It is important to emphasise that the statutory requirement for schools to report pupils’ progress in all 'subjects, AOLs and activities studied... including all national curriculum subjects’ has not changed (see page 25 of the National Data Collection and Reporting Arrangements 2016/17). Consequently, you are required to report to parents based on the other subjects -- the latest announcement only concerns what you do with comments about the LNF.

Many of our schools report the LNF across the whole curriculum – within and alongside both the ‘core’ and ‘non-core’ subjects. What’s changing is that schools are now only required to report the LNF along with English, Welsh and Maths (LLC and MD in FP).

However, the statement ends by stating that ‘if schools feel that they have effective systems in place for reporting on the LNF across the curriculum they are free to continue reporting in this way’. Therefore, whether you chose to embed the LNF within English, Welsh and Maths or to include it in a separate section of your reports, you are meeting the requirements and don’t need to change anything.

​Strand 1 of the curriculum design process was focused on ‘strategic design’. Pioneer Schools were assigned to one of four working groups: Assessment and Progression; Cross-curriculum Responsibilities; Enrichment and Experiences; and Welsh Dimension, International Perspective and Wider Skills. The intention was for the groups to work with expert advisors to address the ‘big questions’ of overall curriculum design. The ‘interim reports’ from the working groups were published at the end of March 2017 (though work began on Strand 2 of the design process in January). In what follows we’ll be looking at the Assessment and Progression group’s report, as it is most relevant to our focus here.

The report from the Assessment and Progression working group outlines ‘principles of assessment and progression for the new curriculum and assessment arrangements’. In particular, they focussed on proposals relating to Recommendation 37 in Successful Futures – that ‘Assessment arrangements should give priority to their formative role in teaching and learning’. The paper summarises the initial findings of the group and their implications for the design and development of the AoLEs. It begins with an explanation of formative assessment and the benefits it offers, before exploring how it can be embedded in the new curriculum. The final section of the paper summarises the development process that produced the report.

The working group makes only a few general recommendations for curriculum design, instead focusing more on embedding formative classroom practices through Initial Teacher Education and Training (ITET) and CPD, school leadership and appropriate accountability measures.

​Section 1 of the interim report underscores Donaldson’s emphasis on the interrelation of pedagogy and assessment, stating that a ‘strong and comprehensive understanding of effective assessment in our schools needs to be at the heart of effective pedagogy’. The report foregrounds formative assessment practice, asserting that summative assessment should be subservient to it. It describes this in terms of bringing formative and summative assessment ‘into harmony’, citing the OECD’s concerns about the distortions that occur when accountability is the primary driver of assessment policy. Following Professor Wynne Harlen’s view that summative assessment should be based on evidence elicited from formative processes, the report argues that education policy should ‘view assessment as a formative process which may at points provide summative data that helps move the learner forward’.

Whilst Successful Futures acknowledges the same assessment can be used formatively or summatively, Donaldson clearly explains how the characteristics of formative and summative assessments differ when they are effectively tailored to their respective outcomes. It is perhaps worth questioning how the Assessment and Progression group’s account of the nature of their interrelatedness is compatible with his proposal that schools should employ a wider range of assessment techniques according to their purpose. As Daisy Christodoulou writes:

​Whilst it’s understandable that the working group should want to ensure summative assessment isn’t driven by an accountability regime, it’s questionable whether it could be entirely subsumed by formative practices. This seems strategically unlikely in light of the different functions Donaldson and Christodoulou ascribe.

​The report makes a number of recommendations for the development of the AoLEs in Section 1, including the following:

​The third section of the report also proposes that:

​The proposals above appear to indicate a trajectory away from using assessment data to track pupil progress, not dissimilar to that witnessed in England. However, whilst recommendations 3-5 all reflect aspects of Donaldson’s proposals, there are instances in Successful Futures that could be in tension with these ideas. For example:

This tension is arguably unresolved within the Donaldson report itself. On the one hand the ‘Achievement Outcomes… have been set deliberately at three-yearly intervals so as to allow teachers to plan and assess learning without constant reference to externally determined criteria’, whereas on the other hand they’re expected to regularly report to parents with reference to the Achievement Outcomes. It could be argued that these potentially conflicting sentiments support Philip Dixon’s concern that people can find their preferences affirmed in Successful Futures, regardless of which perspective they most identify with.

Nowhere in the Donaldson Report is the current practice of aggregating teacher assessment data to track progress questioned as directly as in the principles from Section 3 of the Assessment and Progression group’s report quoted earlier. However, whilst these statements appear in the summary of the group’s recommendations published in the January 2017 edition of the ‘Strategic Stakeholder Newsletter’, in the actual report itself (also released to schools in January) they are prefaced with an illuminating caveat – namely that they record the ‘early thinking’ of the group and have been included ‘for completion’. It seems likely that the proposals from Section 1 are intended to reflect the more mediating approach advanced by Donaldson, thus mitigating the fervour of their early proposals. The latter are arguably more reflective of the English campaign against national curriculum levels. This revisionary pattern can be seen elsewhere in the report, where strong statements in Section 3 such as ‘No extra assessment frameworks will exist. It isn’t useful to have an assessment framework, an AoLE framework plus a DCF and an LNF and a wider skills framework’ are redacted to ‘Additional frameworks sitting outside the AoLE should be kept to a minimum…’ in Section 1 of the report.

It may have been the case that in the earlier stages of Strand 1, some of the group sought to address the drawbacks associated with best-fit assessment using levels (see Part 2 of this series for more detail) by advocating a full-scale move away from this model. Yet the final proposals are considerably more reserved, perhaps because the Progression Steps/Achievement Outcomes are yet to take shape. In the final analysis the report more or less reaffirms Donaldson’s original specification, leaving the question as to precisely how the new arrangements will mitigate against the distorting tendencies of best-fit models largely unanswered.

Section 2 of the working group’s report focuses on how formative assessment can be embedded in teaching and learning. The model proposed by the group is represented by the following infographic:

​At the heart of the group’s proposals is the idea of cultivating a ‘learning culture’ that prioritises formative assessment and has a ‘common language and understanding… for all stakeholders in education including parents, pupils and policy makers’. The report stresses the role ITET has to play in embedding AfL principles as the foundation of an effective learning culture. A recent OECD report related how the practitioners interviewed had difficulty articulating the AfL programme in their school. Responding to this the working group affirms the OECD’s own proposed model for creating a learning culture that is oriented toward cultivating an effective ‘disposition of mind’, as depicted below:

​The framework puts considerable emphasis on metacognition, or ‘learning to learn’. This is a key aspect of Donaldson’s 12 Pedagogical Principles and aligns with Guy Claxton’s framework for ‘Building Learning Power’. The emphasis placed on this model by the working group may confirm a more skills-based approach (as outlined in Part 1 of this series) as the trajectory for Successful Futures’ implementation, though this isn’t spelt out. Whilst Section 2 of the report emphasises the need to reflect the importance of formative assessment in the design of the new curriculum, it doesn’t explain how this will be achieved.
 
In summary, the working group’s report strikes an encouraging tone and adds strategic weight to some of Donaldson’s proposals on assessment, though it falls short of offering any concrete guidelines for shaping the Progression Steps/Achievement Outcomes. Precisely how the process of their development will correlate with that of the AoLEs remains to be seen.

Our engagement with Successful Futures in this blog series has enabled us to situate Professor Donaldson’s proposals within the broader landscape of curriculum and assessment theory. Whilst it hasn’t yielded a concrete picture of how the new curriculum will develop, it has brought the questions around implementation into sharper relief. With Strand 2 of the curriculum development underway, the AoLEs are now taking shape and could be available in draft as early as June this year.

Kirsty Williams has promised that the Welsh Government’s ‘refreshed’ version of their strategic implementation plan – A Curriculum for Wales, A Curriculum for Life – will appear by the end of spring. We will report back when it’s published, highlighting the key developments. Here at the Assessment Foundation we talk with Pioneer Schools on a daily basis, so we'll also share some of their experiences as to how work is progressing 'on the ground'.

​Principle 7 of Donaldson’s 12 Pedagogical Principles states that ‘Good teaching and learning means employing assessment for learning principles’. This raises questions around the crucial distinction between assessment for learning and assessment of learning. The former is usually referred to as ‘formative’ (identifying pupils’ weaknesses in order to find appropriate next steps), whereas the latter is ‘summative’ (giving an account of what pupils’ have learned). Donaldson observes that the distinction isn’t in the means of the assessment (e.g. verbal questioning vs. a written exam) but rather the use to which it is put – hence the same assessment can be used either formatively or summatively.  Navigating the differing emphases of these two forms of assessment has been the cause of much controversy and confusion in education policy. Indeed, the OECD have demonstrated how all assessment systems they’ve surveyed struggle with the tension between their often-competing functions.

Recommendation 37 in Successful Futures advocates that ‘Assessment arrangements should give priority to their formative role in teaching and learning’. Following the popularisation of formative assessment practices by education professors Paul Black and Dylan Wiliam, the Department for Education (DfE) in England has attempted to implement various Assessment for Learning (AfL) initiatives, albeit with minimal long-term success. Daisy Christodoulou argues this is in fact a direct result of government involvement:

​Donaldson contends this is precisely what’s happened in Wales, supporting the conclusion of a major OECD report on assessment that ‘…high-stakes uses of evaluation and assessment results might lead to distortions in the education process…’, and that ‘…it is important to design the accountability uses of evaluation and assessment results in such a way these undesirable effects are minimised’. Donaldson argues that because teacher assessments have been used explicitly for accountability at the end of the Foundation Phase, Key Stage 2 and Key Stage 3, their reliability is compromised and ‘there can also be serious adverse effects on the curriculum’. Consequently recommendation 68 of Successful Futures proposes that teacher assessment should no longer be reported to the Welsh Government – a proposal understandably applauded by the profession. It’s also worth noting that Kirsty Williams has now explicitly stated her intention to implement recommendation 68 in her response to the Children, Young People and Education Committee in February this year.

The Assessment Reform Group defines Assessment for Learning as ‘the process of seeking and interpreting evidence for use by learners and their teachers to decide where the learners are in their learning, where they need to go and how best to get there’. The removal of teacher assessment as an accountability measure supports Donaldson’s prioritisation of this practice. However, removing the requirement to report evidence of learning to government does not automatically mean that schools will have a better AfL culture in their classrooms. Practitioners will continue to be held accountable to their school leadership and ultimately to Estyn – they will still be required to provide evidence that their pupils are progressing at the expected rate. Some argue that any system which grades pupils with Outcomes and Levels encourages a culture of data-tracking that distorts the education process in the manner of which the OECD warns, although as we shall see, this critique is by no means unproblematic.

​In England the DfE removed National Curriculum levels altogether with the expressed aim of counteracting problematic assessment practices arising from their misapplication. However, levels haven’t been replaced and schools are now expected to devise their own assessment systems for monitoring pupil progress. This has led to a proliferation of systems that essentially offer tracking based on ‘levels with another name’, drawing considerable criticism from various education stakeholders. The recent Education Committee enquiry into primary assessment has illustrated the extent of the confusion schools face concerning assessment policy in England. The situation in Wales is more promising however, in that it offers schools an opportunity to develop a standardised assessment system in partnership, drawing on the expertise of international assessment experts. A key question facing the curriculum ‘Pioneer Schools’ tasked with this is: how will Progression Steps and Achievement Outcomes differ from Outcomes and Levels so that the pitfalls associated with ‘best-fit’ models of assessment can be avoided?

Successful Futures goes some way toward anticipating the ways in which the proposed assessment arrangements will differ from the current system. Donaldson describes learning as ‘akin to an expedition, with stops, detours and spurts’ – consequently, Progression Steps should differ from levels in providing more of a ‘road map’ for children’s learning. Moreover, in contrast to levels that are ‘based on a best-fit judgement of overall attainment in a subject at a specific point in time’, Progression Steps should be seen as a ‘staging post’ rather than a ‘judgement’. They will ‘be reference points and not universal expectations of the performance of all children and young people at fixed points’. Whilst Donaldson’s semantics strongly imply the new measures will avoid the pitfalls associated with levels, it’s not exactly clear from the report how this will be achieved. Especially since, as we would argue, the ‘distorting tendencies’ of assessment data aren’t intrinsic to levels per se, but rather a consequence of their misuse. We know that Progression Steps are to be assigned to the same intervals as the current Phase/Stage transitions and that they’ll be comprised of ‘a range of Achievement Outcomes for each Area of Learning and Experience’, written from the pupils’ point of view. Beyond these specifications, we have scarce concrete information as to how Donaldson’s proposed measures will differ from levels in such a way as to minimise ‘distorting effects’.

Christodoulou argues that the ‘distorting effects’ of best-fit judgements using levels is a problem with descriptor-based assessment more generally. She cites various reasons why it’s difficult to get valid formative data from descriptors. Firstly, they’re based on descriptions of pupils’ performance at different stages, as opposed to an analysis of what we know about how particular skills develop (which is arguably more useful for formative purposes). Moreover, they tend to describe skills in a generic way, so that the descriptors can apply to a range of content that isn’t specified. As we saw earlier, Christodolou argues that the specific detail within each subject ought to be the focus, and that skills need to be broken down into their smaller components to be taught effectively. Finally she contends that because descriptor-based assessment doesn’t distinguish between short-term performance and long-term learning, it doesn’t allow teachers to make valid inferences about whether pupils need further instruction on a given topic. Christodoulou’s suggestions for improving the quality of formative assessments include teaching with textbooks, low-stakes diagnostic testing and ‘comparative judgement’ for assessing pupils’ work.

​Maths Education specialist Christian Bokhove critiques Christodoulou’s appraisal of descriptor-based assessment as failing to distinguish poor implementation of good policy from merely bad policy. Whilst Bokhove acknowledges the drawbacks of descriptor-based systems cited by Christodoulou, he’s less convinced that her proposed alternatives don’t suffer from comparable limitations. Moreover, he highlights the absence of engagement with research aimed at mitigating the drawbacks – something the Assessment Foundation has sought to implement with our assessment tools. Bokhove also sees Christodoulou’s pitting of more generic, descriptive feedback against detailed analysis as a false dichotomy – there’s no reason why good systems wouldn’t employ both. This brings us to a commendable aspect of Donaldson’s assessment proposals. As we’ve seen already, he’s very keen to avoid extremes when it comes to models of assessment that are often polarised by academics. It appears that where descriptor-based assessment has dominated assessment policy, it has been unhelpfully made to serve both formative and summative ends. Donaldson has proposed that teacher assessment not be used as an accountability measure precisely to mitigate against distortions and shift the focus to formative applications, but crucially he calls for the new assessment arrangements to ‘promote the use of a wide range of techniques that are appropriate to their purpose’.

For example, Successful Futures emphasises the importance of both self- and peer-assessment. The former enables children to take more responsibility for their learning and equips them to become lifelong learners who can assess their own progress and discern their next steps. Peer-assessment is useful for pupils because it requires them to cultivate a deeper understanding of the nature of the learning they’re evaluating. Another emerging approach Donaldson commends is online ‘adaptive testing’. This has the advantage of dynamically adjusting the level of difficulty in the sequence of questions to account for patterns in responses, and as such lends itself well to formative applications.
To ensure these and other assessment techniques are applied consistently and coherently, Donaldson supports the OECD’s recommendation that ‘a nationally agreed assessment and evaluation framework’ be developed:

​Having a unified framework for assessment policy across Wales is certainly a laudable prospect. Exactly what this will look like and how prescriptive it should be isn’t specified in Successful Futures. Hopefully the framework will offer clear guidance on mitigating the drawbacks of ‘best-fit’ assessment criteria that Christodoulou has highlighted. Yet as education commentator Phillip Dixon observes, it ‘has become something of a truism… that the problems that Welsh education faces are essentially to do with poor delivery’. With Strand 2 of the curriculum development process underway, the spectre of implementation looms, as we await the first drafts of the AoLEs. 

In the next post we’ll consider some of the outputs of Strand 1 of the curriculum reform process, which is now complete. In particular we’ll examine the ‘interim report’ from the Assessment and Progression working group to see if it offers any advance on the questions raised up to this point.

This is the first in a series of three posts that explore the philosophy behind some of the curriculum reform proposals, in particular those relating to assessment. In this post we begin by summarising the core proposals of the Donaldson report, before considering the relationship between assessment and pedagogy.

In March 2014 the DfES announced a thoroughgoing review of Welsh curriculum provision. Less than a year later in February 2015 the review team, led by Professor Graham Donaldson, published Successful Futures: Independent Review of Curriculum and Assessment Arrangements in Wales. The final report was radical and wide-ranging in scope, containing 68 recommendations for curriculum and assessment reform. The proposals were well-received by the education establishment and the profession alike, as reflected in feedback from the subsequent ‘Great Debate’ consultation. The DfES formally accepted Donaldson’s proposals in full and set out plans for implementation in their education strategy document, A Curriculum for Wales, A Curriculum for Life, published in October 2015.

Professor Donaldson’s call for evidence received over 700 responses which, coupled with disappointing PISA results and the 2014 OECD report on Welsh education, led him to conclude that ‘the current national curriculum and assessment arrangements no longer meet the needs of the children and young people of Wales’. To address this situation the report proposed that a new curriculum be built from the ground up with the help of practitioners, signalling a full-scale departure from the National Curriculum of 1988 which had become ‘overloaded, complicated and, in parts, outdated’.

Donaldson has proposed that the new curriculum be developed based on four core purposes, namely that children and young people develop as:

• ambitious, capable learners, ready to learn throughout their lives; 
• enterprising, creative contributors, ready to play a full part in life and work; 
• ethical, informed citizens of Wales and the world; 
• healthy, confident individuals, ready to lead fulfilling lives as valued members of society.

Donaldson observed how the differing philosophies and approaches that characterised the Foundation Phase and Key Stages have led to problematic transitions that hindered progression. Consequently, he calls for 'a continuum of learning from 3 to 16 without phases and key stages'.

Structurally, the new curriculum is to be comprised of six ‘Areas of Learning and Experience’: Expressive arts; Health and well-being; Humanities; Languages, literacy and communication; Mathematics and numeracy; and Science and technology. These will be underpinned by three ‘Cross-Curriculum Responsibilities’: Literacy, Numeracy and Digital Competence. Donaldson has submitted that Digital Competence ought to be given the same weight as Literacy and Numeracy. This is an encouraging advance on the recommendations of the ICT Steering Group’s 2013 report to Welsh Government that Computing be given ‘core’ status as a ‘fourth science’, given its relationship to the other Science, Technology, Engineering and Mathematics (STEM) disciplines and in light of the emerging vocational landscape.

Of the 68 recommendations put forward by Donaldson, 19 deal with assessment. The report acknowledges the need for substantial reform in this area, citing ‘[dissatisfaction] with current assessment arrangements’ as ‘one of the strongest messages’ received from the call for evidence. The ‘continuum of learning’ outlined in Successful Futures is set to replace the current system of Outcomes and Levels with ‘Progression Steps’, which will relate ‘broadly to expectations at ages 5, 8, 11, 14 and 16’. Donaldson questions both the reliability and the validity of the current ‘best-fit’ approach to assessment using levels, which he contends can only provide limited information about pupils’ achievement at best. ‘Progression Steps’ on the other hand, are to provide more of a ‘road map’ for children’s learning, as opposed to being ‘universal expectations of the performance of all children and young people at fixed points’. Each Progression Step will contain ‘Achievement Outcomes’ that reflect the four curriculum purposes and pupils’ achievement in a broader sense than simply measuring attainment. They will be written from the child’s point of view, using an ‘I can…’ or ‘I have…’ construction, and in language they can understand. Encouraging pupils to engage in self-assessment is a key aspect of Donaldson’s proposals – the impact of this approach is commended by a recent OECD review of assessment practice, as enabling pupils to become ‘self-directed learners’.

Successful Futures makes clear the integral relationship between pedagogy and assessment, stating that ‘[assessment for learning] is as relevant to good teaching and learning as it is to assessment’. This connection between what schools teach and how they assess is explored by Daisy Christodoulou in her recent book Making Good Progress? The future of Assessment for Learning. Christodoulou argues that the effectiveness of assessment for learning hinges on whether a ‘generic-skill’ or a ‘deliberate-practice’ method of teaching is adopted. The former is based on the principle that skills are transferable and can be taught directly, whereas Christodoulou advocates the latter, asserting that skills need to be broken down and the component parts taught without necessarily referencing the targeted skill. Conversely, cognitive scientist Guy Claxton has been a long-stranding proponent of a skills-based method called ‘Building Learning Power’. Claxton contends that teaching should be about exercising pupils’ ‘mental muscle groups’. Consequently, according to Claxton the curriculum should provide a ‘comprehensive mental exercise regime’ that cultivates the capacities necessary to acquire the ‘generic ability to learn’.

Christodoulou and Claxton present two nuanced, contrasting examples of current education theory – but they also represent a decades-long, often-polarised debate as to whether the curriculum should focus on teaching pupils knowledge or skills. Donaldson considers this a false dichotomy and attempts to chart a middle way in Successful Futures:

​Nevertheless, Donaldson’s proposal that the curriculum be organised into AoLEs has inevitably drawn comparisons to the structure of the current Foundation Phase, which is based on pedagogical principles aimed at facilitating exploratory learning more so than direct instruction. However, principles 3 and 4 of Donaldson’s own 12 PPs underscore his determination to pursue a both/and approach:

​Again, Donaldson is keen that what are often branded as ‘traditional’ teaching methods (or ‘direct teaching’) should not be excluded by the incorporation of ‘discovery learning or constructivism’. This striving for a more balanced approach is refreshing, although the question as to whether it can be effectively implemented in practice remains. For example, in his recent overview of Welsh education policy Philip Dixon summarises the views of Donaldson’s more cynical critics as ‘worried [that the report] had been a bit too much ‘all things to all men’.
 
A programme of implementation that devolves the responsibility for writing a curriculum to such a broad and diverse network of stakeholders raises many strategic questions. Crucially, how can a tendency toward the silo mentality be avoided? Can a unified reform agenda be realised when so much rests on interpretation of indeterminate proposals? Let’s hope Donaldson’s continuing strategic involvement as chair of the Independent Advisory Group will ensure the development process stays true to his original vision.

In our next post we will explore the emphasis Donaldson places on formative assessment, and the implications of his proposals for teacher assessment practices and accountability.