We received lots of feedback and suggestions at our training events last year, and so Incerts is changing to reflect what teachers need from the system.
Foundation Phase teachers across Wales have asked us to introduce ‘developing steps’ for the Outcome descriptors – the FPP already had “beginning to…” steps, but from the end of Friday 2nd February we’re adding “developing the ability to…” steps too.
FP teachers told us that there is too big a jump from ‘beginning’ to ‘able’ – there is a chasm of difference between a pupil starting to demonstrate a skill, to them being able to do it consistently and in a range of contexts. As a result of this change, teachers will be able to record a more precise picture of what pupils are able to do.
From a planning and teaching perspective, the developing steps will also provide more appropriate next steps for individual pupils, and for groups of pupils with common next steps. Please see Making the Most of your Assessments for details about how to find pupils’ next steps.
The introduction of developing steps will also benefit your pupil progress tracking. Every tick has a contribution to a pupil’s Outcome, so the additional steps allow you to demonstrate smaller steps of progress.
Will I need to re-assess all my pupils?
No. All your previous assessments are still recorded, the developing steps simply give you more options going forward.
Will pupils’ scores change?
Again, no. Where a skill ladder is ticked up to an ‘able’ step, all the earlier ‘developing’ steps will be automatically populated without affecting scores. If only ‘beginning’ is ticked in a skill ladder, ‘developing’ won’t be automatically ticked because we don’t know the degree to which the pupil has achieved it (but you can of course tick them next time you assess).
Another improvement is that we’ll add the Welsh Government’s picture-guidance to the FPP statements. The written guidance (explanations and examples to help make your judgements) were on Incerts already, but we’re adding the visual guidance too in Physical Development.
We believe these changes will primarily benefit teachers’ formative assessment of the FPP. But rest assured that the FPP and the wider Incerts system will continue to be improved, to help and support the teachers using it.
After 10 wonderful years as Chief Executive of the Assessment Foundation, it’s with great sadness that I’ll be stepping down on the 31st January.
As many of you know, I’ve been splitting my time between San Francisco and the UK since getting married in 2015. It’s certainly been a challenging experiment in transatlantic leadership but as my wife and I are expecting our first child on 1st February, it’s now time for me to take on a new role: becoming a father.
I’ll miss you all. It’s been immensely satisfying to work with such a committed and talented group of people, both my colleagues at our HQ and the teachers and schools leaders at our partner schools across the UK. I’ll cherish those memories. I’m happy to announce that I leave the Assessment Foundation in good hands: Dr Philip Dixon, former director of ATL and author of Testing Times, will succeed me as Chief Executive.
The Assessment Foundation can and must do so much more at this pivotal time. Assessment practice is at a crossroads as curriculum reform advances and we should forge a better path for schools. To do so, we must all remember our true north: improving children’s learning. I believe the Assessment Foundation’s brightest and most innovative days are ahead of it. I look forward to watching its success.
Chris Padden, Chief Executive
How have schools engaged with Successful Futures so far?
It’s been encouraging to hear the diverse ways in which Successful Futures’ philosophy is already impacting teaching and learning. Some schools talked about developing ‘rich tasks’ and embedding the three ‘Cross-Curriculum Responsibilities’. Others emphasised aligning their teaching strategy with the ‘Four Purposes’ and ‘12 Pedagogical Principles’. Developing skills-led approaches was a recurring theme, along with applying Foundation Phase pedagogy through Key Stage 2. Some schools emphasised ‘pupil voice’, citing examples of pupil-led, project-based learning, whereas others talked in terms of delivering experiences – what Successful Futures calls ‘authentic contexts’. Several schools acknowledged the importance of self-directed learning, with one school trialling ‘learning passports’ that enable their pupils to assess themselves against the Four Purposes. All Pioneer Schools welcomed what they understood to be a decisive move away from a very rigid, structured approach to teaching based on a highly prescriptive curriculum.
How has the experience of the reform process been for Pioneer Schools?
There was a general consensus that Strand 1 of the development process lacked some clarity of purpose and was slow to get going. By contrast, reports about the progress of Strand 2 were more affirmative. This resonates with Kirsty Williams’ response to the Children and Young People’s Education committee review earlier this year.
Strand 1 of Successful Futures implementation focused on addressing the ‘big questions’ of overall curriculum design. Pioneer Schools were assigned to one of four working groups, each with different responsibilities (Assessment and Progression; Cross-curriculum Responsibilities; Enrichment and Experiences; and Welsh Dimension, International Perspective and Wider Skills). The level of engagement with this Strand ranged from full participation with significant contributions to the final report, through disillusionment with the focus of meetings to withdrawal from the phase altogether. Equally, across both Pioneer and non-Pioneer settings the groups’ final reports received mixed responses. Some schools felt they had consolidated the strategic trajectory for the reforms, whilst others expected them to contain more information about what the new curriculum will look like. Participants in the Cross-Curriculum Responsibilities group were initially dismayed that the form of the LNF and DCF wasn’t expected to change going forward, though their final report seems to suggest Welsh Government have relented on this point.
For Strand 2 the Pioneer Schools have been divided up into six working groups – one for each of the AoLEs. Schools we spoke to were generally in agreement that the momentum of Strand 2 has made a considerable advance on Strand 1. Using the working groups’ reports as a foundation, meetings have been more solution-focussed and clearer on expectations, providing schools with engaging homework to trial in their settings. All schools we spoke to felt confident of this process and excited about the results. As one Headteacher put it, having specifically appointed class teachers as ‘Pioneer Practitioners’ has lent credibility to the development process, because staff are receiving direction from peers who share their workload concerns.
Do schools feel informed about Successful Futures’ implementation?
Feedback from both Pioneer and non-Pioneer schools around communication was largely determined by the configuration of their local clusters, the level of input from their LA or Consortium and how proactive they’ve been as a school. One Pioneer School told us there hadn’t been enough school-to-school networking around Successful Futures, and that schools hadn’t engaged during the first year because no one was particularly clear on the development process. Conversely, another school told us how they had helped organise a conference with Graham Donaldson to bring Swansea and Neath Port Talbot schools up-to-date, with a focus on informing non-Pioneer Schools.
Some schools praised the proactivity of their Consortium and spoke particularly highly of their interactions with the AoLE leads. By contrast, others felt their Consortia had not communicated information very effectively at all. However, a recurring message we received emphasised the fact that the information is available, it’s just that the onus is on schools and their staff to take responsibility for accessing it. For example, one non-Pioneer Headteacher explained how between the Dsyg newsletter, HwB, social media and various initiatives run by their Consortium, information about Successful Futures was rife.
Where is the curriculum development process at now?
Earlier this year Kirsty Williams conceded that Strand 2 of the curriculum development, originally scheduled to begin in September 2016, had been delayed until January. In spite of this, Welsh Government were intent on keeping to their original June deadline for the second phase, meaning schools would have only six months to work on the development of the AoLEs. Most of the Pioneer Schools we spoke with felt this timescale was highly ambitious, although one school did tell us their AoLE group had completed their Strand 2 report on time. However, the timeline for development recently disappeared from the Welsh Government’s website unannounced, and Kirsty Williams’ ‘refreshed’ edition of her department’s implementation plans promised this spring is yet to appear. Whether the draft AoLEs will materialise imminently, as originally scheduled remains to be seen. Reflecting on the fact the Donaldson-led Curriculum for Excellence took a decade to establish in Scotland, education commentator Philip Dixon forecasts that we should ‘Expect to see slippage in terms of delivery dates’. However, it’s undoubtedly more important to get the curriculum reforms right than to compromise on quality for the sake of meeting a deadline.
Putting questions of timescale aside, it was encouraging to hear how the AoLEs are taking shape and being trialled in Pioneer Schools. Now that their development is underway a clear theme seems to be emerging around a much less prescriptive curriculum with a focus on specifying skills over content. In particular it looks like the specification in the AoLEs will be generic enough to apply across the range of disciplines each area encompasses. A recent blog post from the Chairs of the Humanities AoLE group seems to confirm this trajectory:
Within each Area of Learning and Experience a number of ‘traditional’ academic disciplines are being brought together under an overarching umbrella, but what does this mean for these once discretely taught subject disciplines? Without wishing to pre-empt the ongoing development process, it’s likely that within and across the AoLEs the new curriculum will promote interdisciplinarity across subject areas to deepen pupils understanding.
Whilst the feedback on the work of Strand 2 has been positive overall, some schools did raise concerns about how the work of the various groups will be coordinated, given how the six working groups are operating largely independently. There seemed to be a consensus amongst the schools we spoke to that this is yet to be fully ironed out – hopefully Strand 3 will resolve this going forward.
How are the assessment arrangements for the new curriculum progressing?
A key strategic element of the Welsh Government’s plan for implementing the new curriculum is the development of a new ‘Assessment and Evaluation Framework’. In line with the recommendations of the OECD, Donaldson proposed a framework that would ‘define the various roles of pupil assessment and school evaluation’ and ‘align curriculum, teaching and assessment around key learning goals’. Accordingly, the framework should mitigate the confusion surrounding existing assessment and evaluation policies. Whilst work was due to begin on this document from December 2015 in consultation with Pioneer Schools, none of the Pioneers we spoke to were aware of the framework’s existence as a discrete entity.
In her response to the CYPE committee findings, Kirsty Williams emphasised her commitment to ‘carrying out reform of the curriculum and our assessment arrangements in tandem’. At present, all we know about the development of the ‘Progression Steps’ and ‘Achievement Outcomes’ envisioned by Donaldson is that the universities of Wales Trinity Saint David and Glasgow have been handed this responsibility. Whilst it’s understood that experts from these institutions will be working alongside Pioneer Schools to develop the new assessment criteria, curriculum Pioneers we heard from had not been part of these discussions to date. Moreover, the EAS blog referred to this work as still ‘in its exploratory stages’, so perhaps this is to be expected.
It’s been exciting for us to hear from schools across Wales about the various imaginative ways they’ve been engaging with the proposals set out in Successful Futures. The collaborative work of both Pioneer and non-Pioneer Schools has been particularly inspiring. It’s fantastic that all the schools we spoke with are so passionately engaged with delivering the best possible learning experience for their pupils. On the whole, the work of Pioneer Schools on the AoLEs seems to be progressing well, though questions remain as to how the practical issues surrounding the silo-structure of the working groups will be resolved.
At the Assessment Foundation, we remain committed to keeping schools informed and supporting them through changes to curriculum and assessment policy. We also encourage practitioners to take advantage of the information available through the Curriculum for Wales blog, HwB, the Dsyg Newsletter and Consortia updates. We’re also very passionate about hearing your views, so if you have any reflections on the reform process that you would like to share, please do get in touch.
Jane Miller and Finola Wilson (Directors of Impact School Improvement) are delivering Are you ready for Successful Futures? Practical implications of the Donaldson Review at our annual Leadership Conference in Cardiff on Friday 14th July. They’ll pinpoint some key preparation actions that schools should be taking now, ready for the delivery of the new curriculum documents in September 2018. You can check out all the details and book tickets at:
Schools across Wales have come to us with misunderstandings about this recent WG announcement. The page on Learning Wales is entitled “Reporting on the National Literacy and Numeracy Framework is changing” – this is important because the announcement is about the reporting of the Literacy and Numeracy Framework specifically.
It is important to emphasise that the statutory requirement for schools to report pupils’ progress in all 'subjects, AOLs and activities studied... including all national curriculum subjects’ has not changed (see page 25 of the National Data Collection and Reporting Arrangements 2016/17). Consequently, you are required to report to parents based on the other subjects -- the latest announcement only concerns what you do with comments about the LNF.
Many of our schools report the LNF across the whole curriculum – within and alongside both the ‘core’ and ‘non-core’ subjects. What’s changing is that schools are now only required to report the LNF along with English, Welsh and Maths (LLC and MD in FP).
However, the statement ends by stating that ‘if schools feel that they have effective systems in place for reporting on the LNF across the curriculum they are free to continue reporting in this way’. Therefore, whether you chose to embed the LNF within English, Welsh and Maths or to include it in a separate section of your reports, you are meeting the requirements and don’t need to change anything.
The Assessment and Progression Group Reports Back
Strand 1 of the curriculum design process was focused on ‘strategic design’. Pioneer Schools were assigned to one of four working groups: Assessment and Progression; Cross-curriculum Responsibilities; Enrichment and Experiences; and Welsh Dimension, International Perspective and Wider Skills. The intention was for the groups to work with expert advisors to address the ‘big questions’ of overall curriculum design. The ‘interim reports’ from the working groups were published at the end of March 2017 (though work began on Strand 2 of the design process in January). In what follows we’ll be looking at the Assessment and Progression group’s report, as it is most relevant to our focus here.
The report from the Assessment and Progression working group outlines ‘principles of assessment and progression for the new curriculum and assessment arrangements’. In particular, they focussed on proposals relating to Recommendation 37 in Successful Futures – that ‘Assessment arrangements should give priority to their formative role in teaching and learning’. The paper summarises the initial findings of the group and their implications for the design and development of the AoLEs. It begins with an explanation of formative assessment and the benefits it offers, before exploring how it can be embedded in the new curriculum. The final section of the paper summarises the development process that produced the report.
The working group makes only a few general recommendations for curriculum design, instead focusing more on embedding formative classroom practices through Initial Teacher Education and Training (ITET) and CPD, school leadership and appropriate accountability measures.
Formative and Summative Assessment in Harmony?
Section 1 of the interim report underscores Donaldson’s emphasis on the interrelation of pedagogy and assessment, stating that a ‘strong and comprehensive understanding of effective assessment in our schools needs to be at the heart of effective pedagogy’. The report foregrounds formative assessment practice, asserting that summative assessment should be subservient to it. It describes this in terms of bringing formative and summative assessment ‘into harmony’, citing the OECD’s concerns about the distortions that occur when accountability is the primary driver of assessment policy. Following Professor Wynne Harlen’s view that summative assessment should be based on evidence elicited from formative processes, the report argues that education policy should ‘view assessment as a formative process which may at points provide summative data that helps move the learner forward’.
Whilst Successful Futures acknowledges the same assessment can be used formatively or summatively, Donaldson clearly explains how the characteristics of formative and summative assessments differ when they are effectively tailored to their respective outcomes. It is perhaps worth questioning how the Assessment and Progression group’s account of the nature of their interrelatedness is compatible with his proposal that schools should employ a wider range of assessment techniques according to their purpose. As Daisy Christodoulou writes:
It may be possible to make some formative inferences from assessments that have been designed for summative purposes, but they will be limited and imperfect in comparison to assessments that have been designed specifically for formative purposes. Similarly, trying to make summative inferences from tasks that have been designed for formative use is hard to do reliably without sacrificing the flexibility and responsiveness of such tasks.
Whilst it’s understandable that the working group should want to ensure summative assessment isn’t driven by an accountability regime, it’s questionable whether it could be entirely subsumed by formative practices. This seems strategically unlikely in light of the different functions Donaldson and Christodoulou ascribe.
A Future for Data-Tracking?
The report makes a number of recommendations for the development of the AoLEs in Section 1, including the following:
3. All progression statements to be framed in qualitative not quantitate statements which must be accessible to children, teachers and parents. This will ensure the focus for assessment remains formative with learning made ‘visible’ promoting self and peer assessment.
4. Progression statements in both AoLEs and additional frameworks should reflect the ‘road map’ approach described in successful future [sic], they should not be ‘universal expectations of the performance of all children and young people at fixed points.’ (Recommendation 14). Subsequently they should be written as a continuum of skills rather than being age group specific.
5. In addition, these Progression Steps are not set markers at which schools should formally assess all pupils. It is not necessary, nor desirable, to undertake the wholesale, summative assessment of classes or cohorts of pupils at these particular points.
The third section of the report also proposes that:
The proposals above appear to indicate a trajectory away from using assessment data to track pupil progress, not dissimilar to that witnessed in England. However, whilst recommendations 3-5 all reflect aspects of Donaldson’s proposals, there are instances in Successful Futures that could be in tension with these ideas. For example:
Teachers and leaders at all levels need regular information from the assessment recording system to track the progress of individuals and groups of children and young people.
The OECD sees a number of apparent strengths in the current accountability arrangements in Wales… [including] the availability of comprehensive performance data. An additional strength of the system is that the performance data is set in context in that it compares performance with similar schools. The Review’s proposals aim to build on these strengths.
Assessment data must, of course, inform broader evaluations of how well a school is serving its children and young people.
Parents and carers need regular information to find out how well their children are doing and show how they can support them in improving their learning. The Achievement Outcomes and Progression Steps will provide the context for this reporting…
Teachers need to have straightforward ways of tracking individual children and young people’s progress in ways that show progress over time and across the curriculum…
This tension is arguably unresolved within the Donaldson report itself. On the one hand the ‘Achievement Outcomes… have been set deliberately at three-yearly intervals so as to allow teachers to plan and assess learning without constant reference to externally determined criteria’, whereas on the other hand they’re expected to regularly report to parents with reference to the Achievement Outcomes. It could be argued that these potentially conflicting sentiments support Philip Dixon’s concern that people can find their preferences affirmed in Successful Futures, regardless of which perspective they most identify with.
Nowhere in the Donaldson Report is the current practice of aggregating teacher assessment data to track progress questioned as directly as in the principles from Section 3 of the Assessment and Progression group’s report quoted earlier. However, whilst these statements appear in the summary of the group’s recommendations published in the January 2017 edition of the ‘Strategic Stakeholder Newsletter’, in the actual report itself (also released to schools in January) they are prefaced with an illuminating caveat – namely that they record the ‘early thinking’ of the group and have been included ‘for completion’. It seems likely that the proposals from Section 1 are intended to reflect the more mediating approach advanced by Donaldson, thus mitigating the fervour of their early proposals. The latter are arguably more reflective of the English campaign against national curriculum levels. This revisionary pattern can be seen elsewhere in the report, where strong statements in Section 3 such as ‘No extra assessment frameworks will exist. It isn’t useful to have an assessment framework, an AoLE framework plus a DCF and an LNF and a wider skills framework’ are redacted to ‘Additional frameworks sitting outside the AoLE should be kept to a minimum…’ in Section 1 of the report.
It may have been the case that in the earlier stages of Strand 1, some of the group sought to address the drawbacks associated with best-fit assessment using levels (see Part 2 of this series for more detail) by advocating a full-scale move away from this model. Yet the final proposals are considerably more reserved, perhaps because the Progression Steps/Achievement Outcomes are yet to take shape. In the final analysis the report more or less reaffirms Donaldson’s original specification, leaving the question as to precisely how the new arrangements will mitigate against the distorting tendencies of best-fit models largely unanswered.
A Learning Culture
Section 2 of the working group’s report focuses on how formative assessment can be embedded in teaching and learning. The model proposed by the group is represented by the following infographic:
At the heart of the group’s proposals is the idea of cultivating a ‘learning culture’ that prioritises formative assessment and has a ‘common language and understanding… for all stakeholders in education including parents, pupils and policy makers’. The report stresses the role ITET has to play in embedding AfL principles as the foundation of an effective learning culture. A recent OECD report related how the practitioners interviewed had difficulty articulating the AfL programme in their school. Responding to this the working group affirms the OECD’s own proposed model for creating a learning culture that is oriented toward cultivating an effective ‘disposition of mind’, as depicted below:
The framework puts considerable emphasis on metacognition, or ‘learning to learn’. This is a key aspect of Donaldson’s 12 Pedagogical Principles and aligns with Guy Claxton’s framework for ‘Building Learning Power’. The emphasis placed on this model by the working group may confirm a more skills-based approach (as outlined in Part 1 of this series) as the trajectory for Successful Futures’ implementation, though this isn’t spelt out. Whilst Section 2 of the report emphasises the need to reflect the importance of formative assessment in the design of the new curriculum, it doesn’t explain how this will be achieved.
In summary, the working group’s report strikes an encouraging tone and adds strategic weight to some of Donaldson’s proposals on assessment, though it falls short of offering any concrete guidelines for shaping the Progression Steps/Achievement Outcomes. Precisely how the process of their development will correlate with that of the AoLEs remains to be seen.
The Road Ahead
Our engagement with Successful Futures in this blog series has enabled us to situate Professor Donaldson’s proposals within the broader landscape of curriculum and assessment theory. Whilst it hasn’t yielded a concrete picture of how the new curriculum will develop, it has brought the questions around implementation into sharper relief. With Strand 2 of the curriculum development underway, the AoLEs are now taking shape and could be available in draft as early as June this year.
Kirsty Williams has promised that the Welsh Government’s ‘refreshed’ version of their strategic implementation plan – A Curriculum for Wales, A Curriculum for Life – will appear by the end of spring. We will report back when it’s published, highlighting the key developments. Here at the Assessment Foundation we talk with Pioneer Schools on a daily basis, so we'll also share some of their experiences as to how work is progressing 'on the ground'.
Principle 7 of Donaldson’s 12 Pedagogical Principles states that ‘Good teaching and learning means employing assessment for learning principles’. This raises questions around the crucial distinction between assessment for learning and assessment of learning. The former is usually referred to as ‘formative’ (identifying pupils’ weaknesses in order to find appropriate next steps), whereas the latter is ‘summative’ (giving an account of what pupils’ have learned). Donaldson observes that the distinction isn’t in the means of the assessment (e.g. verbal questioning vs. a written exam) but rather the use to which it is put – hence the same assessment can be used either formatively or summatively. Navigating the differing emphases of these two forms of assessment has been the cause of much controversy and confusion in education policy. Indeed, the OECD have demonstrated how all assessment systems they’ve surveyed struggle with the tension between their often-competing functions.
Recommendation 37 in Successful Futures advocates that ‘Assessment arrangements should give priority to their formative role in teaching and learning’. Following the popularisation of formative assessment practices by education professors Paul Black and Dylan Wiliam, the Department for Education (DfE) in England has attempted to implement various Assessment for Learning (AfL) initiatives, albeit with minimal long-term success. Daisy Christodoulou argues this is in fact a direct result of government involvement:
When government get their hands on anything involving the word ‘assessment’, they want it to be about high-stakes monitoring and tracking, not low-stakes diagnostics. That is, the involvement of government in AfL meant that the assessment in AfL went from being formative to being summative…
Donaldson contends this is precisely what’s happened in Wales, supporting the conclusion of a major OECD report on assessment that ‘…high-stakes uses of evaluation and assessment results might lead to distortions in the education process…’, and that ‘…it is important to design the accountability uses of evaluation and assessment results in such a way these undesirable effects are minimised’. Donaldson argues that because teacher assessments have been used explicitly for accountability at the end of the Foundation Phase, Key Stage 2 and Key Stage 3, their reliability is compromised and ‘there can also be serious adverse effects on the curriculum’. Consequently recommendation 68 of Successful Futures proposes that teacher assessment should no longer be reported to the Welsh Government – a proposal understandably applauded by the profession. It’s also worth noting that Kirsty Williams has now explicitly stated her intention to implement recommendation 68 in her response to the Children, Young People and Education Committee in February this year.
The Assessment Reform Group defines Assessment for Learning as ‘the process of seeking and interpreting evidence for use by learners and their teachers to decide where the learners are in their learning, where they need to go and how best to get there’. The removal of teacher assessment as an accountability measure supports Donaldson’s prioritisation of this practice. However, removing the requirement to report evidence of learning to government does not automatically mean that schools will have a better AfL culture in their classrooms. Practitioners will continue to be held accountable to their school leadership and ultimately to Estyn – they will still be required to provide evidence that their pupils are progressing at the expected rate. Some argue that any system which grades pupils with Outcomes and Levels encourages a culture of data-tracking that distorts the education process in the manner of which the OECD warns, although as we shall see, this critique is by no means unproblematic.
A New Model for Tracking Progress?
In England the DfE removed National Curriculum levels altogether with the expressed aim of counteracting problematic assessment practices arising from their misapplication. However, levels haven’t been replaced and schools are now expected to devise their own assessment systems for monitoring pupil progress. This has led to a proliferation of systems that essentially offer tracking based on ‘levels with another name’, drawing considerable criticism from various education stakeholders. The recent Education Committee enquiry into primary assessment has illustrated the extent of the confusion schools face concerning assessment policy in England. The situation in Wales is more promising however, in that it offers schools an opportunity to develop a standardised assessment system in partnership, drawing on the expertise of international assessment experts. A key question facing the curriculum ‘Pioneer Schools’ tasked with this is: how will Progression Steps and Achievement Outcomes differ from Outcomes and Levels so that the pitfalls associated with ‘best-fit’ models of assessment can be avoided?
Successful Futures goes some way toward anticipating the ways in which the proposed assessment arrangements will differ from the current system. Donaldson describes learning as ‘akin to an expedition, with stops, detours and spurts’ – consequently, Progression Steps should differ from levels in providing more of a ‘road map’ for children’s learning. Moreover, in contrast to levels that are ‘based on a best-fit judgement of overall attainment in a subject at a specific point in time’, Progression Steps should be seen as a ‘staging post’ rather than a ‘judgement’. They will ‘be reference points and not universal expectations of the performance of all children and young people at fixed points’. Whilst Donaldson’s semantics strongly imply the new measures will avoid the pitfalls associated with levels, it’s not exactly clear from the report how this will be achieved. Especially since, as we would argue, the ‘distorting tendencies’ of assessment data aren’t intrinsic to levels per se, but rather a consequence of their misuse. We know that Progression Steps are to be assigned to the same intervals as the current Phase/Stage transitions and that they’ll be comprised of ‘a range of Achievement Outcomes for each Area of Learning and Experience’, written from the pupils’ point of view. Beyond these specifications, we have scarce concrete information as to how Donaldson’s proposed measures will differ from levels in such a way as to minimise ‘distorting effects’.
Do Descriptor-based Criteria Distort Assessment Data?
Christodoulou argues that the ‘distorting effects’ of best-fit judgements using levels is a problem with descriptor-based assessment more generally. She cites various reasons why it’s difficult to get valid formative data from descriptors. Firstly, they’re based on descriptions of pupils’ performance at different stages, as opposed to an analysis of what we know about how particular skills develop (which is arguably more useful for formative purposes). Moreover, they tend to describe skills in a generic way, so that the descriptors can apply to a range of content that isn’t specified. As we saw earlier, Christodolou argues that the specific detail within each subject ought to be the focus, and that skills need to be broken down into their smaller components to be taught effectively. Finally she contends that because descriptor-based assessment doesn’t distinguish between short-term performance and long-term learning, it doesn’t allow teachers to make valid inferences about whether pupils need further instruction on a given topic. Christodoulou’s suggestions for improving the quality of formative assessments include teaching with textbooks, low-stakes diagnostic testing and ‘comparative judgement’ for assessing pupils’ work.
Maths Education specialist Christian Bokhove critiques Christodoulou’s appraisal of descriptor-based assessment as failing to distinguish poor implementation of good policy from merely bad policy. Whilst Bokhove acknowledges the drawbacks of descriptor-based systems cited by Christodoulou, he’s less convinced that her proposed alternatives don’t suffer from comparable limitations. Moreover, he highlights the absence of engagement with research aimed at mitigating the drawbacks – something the Assessment Foundation has sought to implement with our assessment tools. Bokhove also sees Christodoulou’s pitting of more generic, descriptive feedback against detailed analysis as a false dichotomy – there’s no reason why good systems wouldn’t employ both. This brings us to a commendable aspect of Donaldson’s assessment proposals. As we’ve seen already, he’s very keen to avoid extremes when it comes to models of assessment that are often polarised by academics. It appears that where descriptor-based assessment has dominated assessment policy, it has been unhelpfully made to serve both formative and summative ends. Donaldson has proposed that teacher assessment not be used as an accountability measure precisely to mitigate against distortions and shift the focus to formative applications, but crucially he calls for the new assessment arrangements to ‘promote the use of a wide range of techniques that are appropriate to their purpose’.
For example, Successful Futures emphasises the importance of both self- and peer-assessment. The former enables children to take more responsibility for their learning and equips them to become lifelong learners who can assess their own progress and discern their next steps. Peer-assessment is useful for pupils because it requires them to cultivate a deeper understanding of the nature of the learning they’re evaluating. Another emerging approach Donaldson commends is online ‘adaptive testing’. This has the advantage of dynamically adjusting the level of difficulty in the sequence of questions to account for patterns in responses, and as such lends itself well to formative applications.
To ensure these and other assessment techniques are applied consistently and coherently, Donaldson supports the OECD’s recommendation that ‘a nationally agreed assessment and evaluation framework’ be developed:
In line with the wider Review proposals, the framework should ‘…aim to align curriculum, teaching and assessment around key learning goals and include a range of different assessment approaches and formats…’. It should be clear about the formative and summative roles of assessment and distinguish between those activities whose place lies in learning and teaching and those that will contribute to self-evaluation, external accountability and national monitoring. In particular, it should explain how the components of the assessment framework address issues of validity and reliability in the methods used.
Having a unified framework for assessment policy across Wales is certainly a laudable prospect. Exactly what this will look like and how prescriptive it should be isn’t specified in Successful Futures. Hopefully the framework will offer clear guidance on mitigating the drawbacks of ‘best-fit’ assessment criteria that Christodoulou has highlighted. Yet as education commentator Phillip Dixon observes, it ‘has become something of a truism… that the problems that Welsh education faces are essentially to do with poor delivery’. With Strand 2 of the curriculum development process underway, the spectre of implementation looms, as we await the first drafts of the AoLEs.
In the next post we’ll consider some of the outputs of Strand 1 of the curriculum reform process, which is now complete. In particular we’ll examine the ‘interim report’ from the Assessment and Progression working group to see if it offers any advance on the questions raised up to this point.
This is the first in a series of three posts that explore the philosophy behind some of the curriculum reform proposals, in particular those relating to assessment. In this post we begin by summarising the core proposals of the Donaldson report, before considering the relationship between assessment and pedagogy.
Successful Futures: A Summary
In March 2014 the DfES announced a thoroughgoing review of Welsh curriculum provision. Less than a year later in February 2015 the review team, led by Professor Graham Donaldson, published Successful Futures: Independent Review of Curriculum and Assessment Arrangements in Wales. The final report was radical and wide-ranging in scope, containing 68 recommendations for curriculum and assessment reform. The proposals were well-received by the education establishment and the profession alike, as reflected in feedback from the subsequent ‘Great Debate’ consultation. The DfES formally accepted Donaldson’s proposals in full and set out plans for implementation in their education strategy document, A Curriculum for Wales, A Curriculum for Life, published in October 2015.
Professor Donaldson’s call for evidence received over 700 responses which, coupled with disappointing PISA results and the 2014 OECD report on Welsh education, led him to conclude that ‘the current national curriculum and assessment arrangements no longer meet the needs of the children and young people of Wales’. To address this situation the report proposed that a new curriculum be built from the ground up with the help of practitioners, signalling a full-scale departure from the National Curriculum of 1988 which had become ‘overloaded, complicated and, in parts, outdated’.
Donaldson has proposed that the new curriculum be developed based on four core purposes, namely that children and young people develop as:
Donaldson observed how the differing philosophies and approaches that characterised the Foundation Phase and Key Stages have led to problematic transitions that hindered progression. Consequently, he calls for 'a continuum of learning from 3 to 16 without phases and key stages'.
Structurally, the new curriculum is to be comprised of six ‘Areas of Learning and Experience’: Expressive arts; Health and well-being; Humanities; Languages, literacy and communication; Mathematics and numeracy; and Science and technology. These will be underpinned by three ‘Cross-Curriculum Responsibilities’: Literacy, Numeracy and Digital Competence. Donaldson has submitted that Digital Competence ought to be given the same weight as Literacy and Numeracy. This is an encouraging advance on the recommendations of the ICT Steering Group’s 2013 report to Welsh Government that Computing be given ‘core’ status as a ‘fourth science’, given its relationship to the other Science, Technology, Engineering and Mathematics (STEM) disciplines and in light of the emerging vocational landscape.
A Mandate for Assessment Reform
Of the 68 recommendations put forward by Donaldson, 19 deal with assessment. The report acknowledges the need for substantial reform in this area, citing ‘[dissatisfaction] with current assessment arrangements’ as ‘one of the strongest messages’ received from the call for evidence. The ‘continuum of learning’ outlined in Successful Futures is set to replace the current system of Outcomes and Levels with ‘Progression Steps’, which will relate ‘broadly to expectations at ages 5, 8, 11, 14 and 16’. Donaldson questions both the reliability and the validity of the current ‘best-fit’ approach to assessment using levels, which he contends can only provide limited information about pupils’ achievement at best. ‘Progression Steps’ on the other hand, are to provide more of a ‘road map’ for children’s learning, as opposed to being ‘universal expectations of the performance of all children and young people at fixed points’. Each Progression Step will contain ‘Achievement Outcomes’ that reflect the four curriculum purposes and pupils’ achievement in a broader sense than simply measuring attainment. They will be written from the child’s point of view, using an ‘I can…’ or ‘I have…’ construction, and in language they can understand. Encouraging pupils to engage in self-assessment is a key aspect of Donaldson’s proposals – the impact of this approach is commended by a recent OECD review of assessment practice, as enabling pupils to become ‘self-directed learners’.
Assessment meets Pedagogy
Successful Futures makes clear the integral relationship between pedagogy and assessment, stating that ‘[assessment for learning] is as relevant to good teaching and learning as it is to assessment’. This connection between what schools teach and how they assess is explored by Daisy Christodoulou in her recent book Making Good Progress? The future of Assessment for Learning. Christodoulou argues that the effectiveness of assessment for learning hinges on whether a ‘generic-skill’ or a ‘deliberate-practice’ method of teaching is adopted. The former is based on the principle that skills are transferable and can be taught directly, whereas Christodoulou advocates the latter, asserting that skills need to be broken down and the component parts taught without necessarily referencing the targeted skill. Conversely, cognitive scientist Guy Claxton has been a long-stranding proponent of a skills-based method called ‘Building Learning Power’. Claxton contends that teaching should be about exercising pupils’ ‘mental muscle groups’. Consequently, according to Claxton the curriculum should provide a ‘comprehensive mental exercise regime’ that cultivates the capacities necessary to acquire the ‘generic ability to learn’.
Christodoulou and Claxton present two nuanced, contrasting examples of current education theory – but they also represent a decades-long, often-polarised debate as to whether the curriculum should focus on teaching pupils knowledge or skills. Donaldson considers this a false dichotomy and attempts to chart a middle way in Successful Futures:
The ‘subject against skill/competence’ debate represents an unhelpful polarisation, since both make important contributions to fulfilling the purposes of the curriculum. The structure of the curriculum should therefore ensure that the vital contribution of disciplinary learning is preserved but is supplemented by other aspects that relate directly to the needs of today and provide sound preparation for the challenges of tomorrow.
Nevertheless, Donaldson’s proposal that the curriculum be organised into AoLEs has inevitably drawn comparisons to the structure of the current Foundation Phase, which is based on pedagogical principles aimed at facilitating exploratory learning more so than direct instruction. However, principles 3 and 4 of Donaldson’s own 12 PPs underscore his determination to pursue a both/and approach:
3) Good teaching and learning means employing a blend of approaches including direct teaching
4) Good teaching and learning means employing a blend of approaches including those that promote problem solving, creative and critical thinking
Again, Donaldson is keen that what are often branded as ‘traditional’ teaching methods (or ‘direct teaching’) should not be excluded by the incorporation of ‘discovery learning or constructivism’. This striving for a more balanced approach is refreshing, although the question as to whether it can be effectively implemented in practice remains. For example, in his recent overview of Welsh education policy Philip Dixon summarises the views of Donaldson’s more cynical critics as ‘worried [that the report] had been a bit too much ‘all things to all men’.
A programme of implementation that devolves the responsibility for writing a curriculum to such a broad and diverse network of stakeholders raises many strategic questions. Crucially, how can a tendency toward the silo mentality be avoided? Can a unified reform agenda be realised when so much rests on interpretation of indeterminate proposals? Let’s hope Donaldson’s continuing strategic involvement as chair of the Independent Advisory Group will ensure the development process stays true to his original vision.
In our next post we will explore the emphasis Donaldson places on formative assessment, and the implications of his proposals for teacher assessment practices and accountability.
Across the border in England, the DfE are in the throes of an enquiry into primary assessment following the removal of National Curriculum Levels. Meanwhile, Welsh education is undergoing an exciting and challenging transition. Education practitioners are faced with a unique opportunity to build an entirely new curriculum from the ground up - the challenge right now is the implementation of this ambitious project.
Moving Forward Together
We were excited to attend the recent Central South Consortium conference “Moving Forward Together”, where we heard from Pioneer and non-Pioneer schools about their engagement with Graham Donaldson’s proposals in his review of the current Welsh curriculum. It was encouraging to see widespread enthusiasm for the “four curriculum purposes” and the “pedagogical principles” outlined in Successful Futures. Equally heartening to hear, were stories about the creative ways schools have been embedding digital competence as a cross-curriculum responsibility.
What was noticeably absent from the conference was an update from the curriculum Pioneer schools about progress on the development of the Areas of Learning and Experience (AoLEs) and the assessment framework(s) for the new curriculum. This is unsurprising in light of the recent findings of the Children, Young People and Education Committee’s (CYPE) enquiry into the implementation of Successful Futures.
The CYPE Committee met between November 2016 and January this year to scrutinise the progress of the Donaldson reforms. They heard evidence from Donaldson himself, various Pioneer schools, the Regional Consortia, teachers’ unions and the Education Secretary, amongst other stakeholders. Arguably the most pressing issue was the delay of Strand 2 of Welsh Government’s schedule for implementation – the “design of [the] framework and principles for each AoLE, including progression reference points and achievement outcomes” – reflected in the committee’s concluding judgement that “some aspects of the implementation of Successful Futures are not progressing as well as we would expect”. Due to begin in September 2016, the Committee heard that the initial meetings for Strand 2 were due to commence at the end of January.
In their evidence submitted to the CYPE Committee, the NAHT, NUT and NASUWT all raised concerns about there being insufficient clarity amongst Pioneer schools about exactly what they had been tasked with. This lack of understanding has led to frustrations for some Pioneers with the reform process generally, as was acknowledged in the report from the Regional Consortia. The challenge will now be for the groups responsible for the strategic oversight of the reforms (the Independent Advisory Group, Change Board, Strategic Stakeholder Group and the recently coined “Programme Board”) to facilitate more effective communication with and between Pioneer schools, in addition to coordinating the development of the new curriculum content.
Teachers’ unions also expressed anxieties about the threat of tribalism emerging from the divide between Pioneer and non-Pioneer schools. The need for unified engagement from the whole sector was a key emphasis of the CSC conference, and is undoubtedly a consideration of the joint proposal the Consortia have put to Welsh Government. In this they “offer a strong, focused and detailed structure for realising the recommendations” of the review. Kirsty Williams has also promised a review of the Pioneer school model itself, due to be published at the end of 2017.
A Demanding Timescale
Whilst the CYPE Committee affirmed the partnership model for curriculum development, they also echoed the NAHT’s concerns that the quality of the work could be compromised by an increasingly demanding timescale – “it must be about getting this right for pupils and not rushing in change to meet a deadline”. The DfES had originally been considering a timescale of 8-10 years, based on the Scottish experience of curriculum reform, but the turnaround was subsequently fixed at three years from the beginning of the Pioneer schools’ work. However, the missed deadline for Strand 1 means the AoLE design work in Strand 2 has to be completed in a considerably smaller window (something the Minister conceded to the Committee). This was confirmed at the beginning of February by the appearance of the third version of the department’s “Plan for curriculum and assessment design and development” on their website, dated “November 2016”. In light of the timescale issues experienced thus far, the Committee proposed that Welsh Government undertake contingency planning in case the Pioneer model fails to deliver the desired output in time.
The DfES released the fourth curriculum “Stakeholder Newsletter” at the end of January, in which they summarise the preliminary output and announce that the first incarnation of the AoLEs should appear by June 2017. The recommendations from the various curriculum working groups are largely constructive and encouraging. In the meantime the Kirsty Williams' response to the Committee’s findings is due any day, following her announcement that the “refreshed” Welsh Government strategic delivery plan (“Qualified for life: A curriculum for Wales – a curriculum for life”) will be released in the spring. The Committee acknowledged the challenge of translating the conceptual Donaldson Review into concrete proposals that would be understood on the ground. Let’s hope the department accepts the Committee’s recommendations and commits to maintaining tangible ways in which this unique and exciting partnership model can move forward.
We should be encouraged by the Education Secretary's determination to "keep the child at the centre and develop a broad, balanced, inclusive and challenging curriculum". The current mandate for reform has the potential to deliver an education system that will enable children and young people in Wales to flourish in the modern world - an agenda we can all get behind.
Find out more - key strategic documents
The Assessment Foundation works alongside the four regional consortia in Wales, because we all want to ensure that our messages to schools are consistent. We are pleased to be working particularly closely with GwE, where almost 90% of primary schools use Incerts for formative assessment, and to support the summative assessment at the ends of the Key Stages.
There has always been inconsistency across Wales in how this summative assessment is made, and GwE have been working to address this problem recently. For the end of Key Stage assessments in summer 2017, they have decided that an "over half" threshold should be used. They have written “to best reflect an outcome or level, a learner must achieve the majority of statements noted against that outcome or level”, and they have sent schools three examples of how to arrive at a whole Outcome or Level this way. However, they have added that end of Key Stage assessments from Incerts and other electronic systems should not be used this year, and we know this has surprised some people.
The reason is that Incerts does not normally produce the whole Outcomes/Levels needed for the statutory End of Key Stage data directly, and if different schools use it in different ways this can lead to inconsistency. Incerts does “count up” all the individual statements achieved by a pupil as GwE described, but it uses them to produce sub-Outcomes and sub-Levels, or decimal Outcomes and decimal Levels. When whole Outcomes or Levels are required, we have always left it to the school to take a closer look at the “c”s, and to decide when to award a “secure Outcome 5” or a “secure Level 4”, etc.
However, you can follow GwE’s “majority of statements” rule and still use Incerts to help you: there’s no need to count statements manually or judge whether enough have been ticked “by eye”. The thresholds built into Incerts are such that a pupil will be in the “top half of the c band” if the majority of statements are achieved, and the “bottom half of the c band” if not. So, to ensure you follow GwE’s rule at the end of this year, you just need to look at the decimal Outcomes/Levels and know that the mid-point of our “c” band is at “X.17”. Put simply, if the decimal score is above “X.17” you can say the Outcome or Level is secure by the “majority of statements” rule. So for example, you would return Level 4 to Welsh Government for Key Stage 2 pupils scoring above 4.17, and so on.
It’s relatively straightforward to make this cut-off automatically in a spreadsheet, and our Support Team will create a template for you shortly. If you have any questions in the meantime though, please don’t hesitate to get in touch.