The first in an occasional series of responses to questions from The Education Hub’s Nina Hood

Hi Nina,

Thanks for this clarification.

Just briefly to address some of your areas of concern:

Relationship with school curriculum

This project is in alignment with the ethos behind the school curriculum. The “Learning Vision” of the school reads:

“It is our intention that the curriculum will continue to be broad, with the opportunity for more personalised choices as students progress through the school. The curriculum emphasises important skills for our students to take to their futures, such as learning how to learn, critical thinking, interactivity and the ability to use technology. Evaluation and assessment is continuous, guiding and formative. Learning occurs in a learner-centred, holistic setting. The importance of student agency and collaborative approaches are emphasised because we want adaptive, creative, responsive students who can manage themselves and direct their own learning.”

Mount Aspiring College Learning Vision. December 2018

Even without illustration you can see how an assessment project like this has the potential to further a range of the objectives of the school. I’ve made bold the obvious ones. Our school starts at Year 7 and runs integrated learning programmes based around a ‘homeroom’ set-up, some in modern learning environments, until the end of Year 9. We also have a strong Outdoor Pursuits thread through the school which builds on the ethos of individual responsibility for self and to others. We have an implementation of the PB4L strategies that we call “SOAR” (Self, Others, Attitude and Respect”) which underpins the pastoral aspect of the school – and we run a formal community volunteering programme which sees students spending 50+ hours in community volunteering activities. The vision we have is that this badge project will allow us to ’surface’ (generally I dislike the ‘verbing’ of words, but hey, two in one paragraph) the aspects of student learning that we value the most, at the same time as having a rigorous, transparent and contestable means of awarding credentials for this valued learning that is something the students can navigate and recruit to their own needs as much as we as teachers use it for ours.

We do envisage this system supplanting many of the conventional means of assessment currently used, with the exception of some of the nationally standardised testing in core learning areas like Reading, which would form a significant reference point for the information our scheme is providing. We also will continue to need to ensure students can be sufficiently skilled in traditional assessment mechanisms like essays and examinations in order to succeed in future tertiary and external assessment settings.

The implementation strategy we have devised borrows from the Spark “design thinking” approach of first defining the user, and we’ve worked together in the initial stages, both individually with team members and as a collective group, to identify and clarify the needs of 4 key user groups: Teachers, Students, Families and the Community and the State (and other stakeholders, like you). In our project team we have representatives of each of these ‘users’ who will be responsible to testing what we’re creating against their identified needs.

I have also developed a ’template’ for the precursor process to designing a badge which also operates around this ‘user’ model, identifying the needs that the innovation meets. I’m also developing a schema for organising the badges which has three layers:

• LAYER A: Curricular Links – Including aspects such as key competencies and the PB4L dispositions. This means that the badges ultimately have to form a matrix that ‘covers’ the curriculum as a whole. As fixed standards, the badges can conceivably all be hooked on to a curriculum area and a curriculum level – so “progress” against the curriculum might be signalled by the system (though, I have some misgivings about the idea that learning should progress in such a ‘linear’ fashion, and as an English teacher I can only say that I think the discriminating features we often claim delineates work between curriculum levels are arbitrary at best).
• LAYER B: Badge Type – In order to develop a ’typology’ that allows for consistency between badges, remembering that these are ’standards-based’ assessments, I have developed 4 classifications: 
  • Knowledge – this is often very specific and can often be strictly relevant to particular subject areas. Often the higher-order work cannot be successfully completed until the basic knowledge is secure – the micro-credentials are a means of ensuring this knowledge is valued and its acquisition rewarded.  There is more than one type of knowledge, of course, and some skills can shift into the knowledge domain when we become automatic in our employment of them – so some knowledge badges can be for highly abstract concepts as much as they can be for concrete information. A key element of the conditions of assessment of knowledge is that the mode of demonstrating knowledge does not necessarily have to be specified in an assessment. It could be presented by the student in speech, writing, dance, sculptural form. 
  • Skill – Often considered to be the ‘application of knowledge’, skill badges will have a different typology because they will often describe a mode of employing knowledge or showing learning. Skills will often have to be practiced over time and can come about as a consequence of focussed training.. memorisation would be a skill, as would speaking be, or simplifying a fraction, or building a dovetail joint. Some skills are unique to a subject domain, but many appear in and are developed by many sites of operation. It would be a goal of this project to develop credentials for skills (like speaking, for example) that could be unlocked by students in many settings. This would also mean developing a common language and understanding of what makes ‘effective speaking’.
  • Disposition – While this is not necessarily something that must be demonstrated or awarded in isolation from other elements of the learning process, there are dispositions, like tenacity, that we would like to develop a mechanism to reward in a student. It’s conceivable, but probably controversial, that we could develop a credential for trying hard to improve at something, and making definite improvement as a result, without the end goal necessarily having been reached. 
  • Performance – I may not yet have developed the best language for these types, but this type refers to a credential that is awarded to a student for enacting the learning in a rich context – it might be achieved as an automatic result of acquiring an array of other more atomised achievements – like a dramatic performance might be made up of micro-credentials from all three of the former types, but equally, in order to show the value we place on ‘putting it all together’ in a meaningful context, these badges may be awarded for significant pieces of work that combine a range of knowledge and skill. Just today my Year 10 class broadcast a live 45 minute investigative documentary on Youtube, which was a culmination of their year’s work. There should be a badge for that.
• LAYER C – Purpose of the scheme – This is the aspect of the badge scheme where badges must be tested against the requirements of each ‘user group’, our knowledge of effective assessment practice, and the degree to which it allows the pursuit of assessment outcomes without making the assessment the point of the learning – all the “making the ‘main thing’ the ‘main thing’”, business – I certainly despair at the possibility that students and families experience school as an institution in pursuit of passing it’s own assessments almost to the exclusion of everything else. This means that a badge can be fun too. It can challenge convention. It occasionally can ‘surface’ something ridiculous just for laughs – and it can look beautiful, arrive instantly and become an object of attraction and desire.

I look forward to talking through an annotated example. I’ve got a few in mind – but again, I think it would be of great benefit for us to do this with members of the wider team. Part of the implementation strategy is that these initial people, who are drawn from a wide area of the school, will become evangelists for the scheme. I feel we need to feed/stimulate them as much as we can.

Here are some initial responses to your further questions below:

• How does the team assure rigour and relevance? 

The methods we have planned involve the triangulation of assessment outcomes with established assessments, the use of moderation panels – applying a rigorous, NCEA-style random sampling of achieved credentials and the publishing of the results. An example of this can be seen here:

http://achieve.community.edutronic.net/stage-two-reading/discovery/

We are also working with student, community and parent ‘advocates’ in the development process to ensure their input into aspects of relevance and usability, credibility are constantly incorporated into our design process.

• How do you decide what to award badges for? and who is involved in these decisions? 

In the initial stage, classroom teachers will devise badges that are designed to provide a means of giving credit to the most important elements of learning that they wish to highlight in their learning programmes. These are developed in conjunction with programmes of learning that are verified by the Heads of Learning Area who keep an eye on curriculum coverage and diversity. We also apply our professional understanding of strong assessment practice both to make judicious decisions about what ’sampling’ of the students’ work takes place via this method and also to maintain a healthy awareness of the limits of such procedures. Sometimes assessments will also be developed in order to ensure students are given a means of adding mana to attainments that are of personal value to them. There may also be a set of badges developed by students and families to affirm teacher performance. As well as the use of badges to provide more rigour to internal professional development processes. It has been known for a parent to attempt a badge or two as well. 

• How, if at all, have assessment processes been adapted to ensure fairness and transparency in awarding the badges? 

I think at this stage I would say that the badges represent an improvement in our (largely class-based) testing regime to increase the level of fairness and transparency in the process – as well as to reduce the over-all assessment burden on teachers and students by ensuring that the means of assessment is simple – and that we are precise in what we test, as opposed to making every assessment, for example, a written language marathon.

• How are these processes the same/different from business as usual at the school 

The differences can be described by much of what precedes this – and in addition to that, the fact that students will be unlocking achievements as they learn, there may also be benefits in the affect domain that arise from the application of some of the more well-known psychological elements of game theory. The fact that there’s a sense of immediate gratification from achieving, the fact that the students are empowered to try to achieve key assessments on multiple occasions, sometimes over multiple settings, the way they are able to capitalise on what they have already learned, and build a portfolio of achievements that allow them to feel proud of what they achieve, and to engage in healthy competition, and the fact that gaps in learning can be very easily identified all make this quite different to the current “here’s your mark, but you move on anyway” approach that is currently employed.

• What challenges have they encountered so far? (What advice would they have for other schools wanting to start out down this path?).

It’s very early stages, but the main challenges, as always, will be time – especially getting people together. I have found the process of individual workshops that precede group meetings to be a useful way to get everyone in the same place before a group exercise, leading to high levels of output from the group exercise. 

In relation to the examination of the impact of this project, we’ve definitely isolated the two areas: Student Achievement and Student Agency as starting points for the lines of inquiry we ‘d most like to develop rigorous investigations into. Your help, and that of Rosemary Hipkins, in narrowing these down and designing effective tools for measuring these will be invaluable.

As we progress, we will document the challenges in more detail.
Is it ok if I publish your questions and my replies on the blog in a “Letters from Nina” section?

Righto!

Chris

Co-Dean of Year 13 and Teacher of English
MOUNT ASPIRING COLLEGE
“Risk! Risk anything! Care no more for the opinions of others, for those voices. Do the hardest thing on earth for you. Act for yourself. Face the truth.”(Katherine Mansfield)

On 5 December 2018 at 19:35:45, Nina Hood (nina@theeducationhub.org.nz) wrote:

Two key areas that we would like to explore further with you.

Firstly, how you are thinking and making decisions in relation to connecting the badges to the curriculum and also what we know about effective assessment practices. 

By our January meeting (or in the Zoom discussions we have before) we would like you to talk us through your planning and curriculum decision-making – and how these process relate to the school’s overall curriculum. Some examples of the documentation used could be useful for us, or you might like to talk us through an annotated example of a badge that you have developed.  

Building on this, below are some questions that we would want you to be thinking about as you start the implementation of our project. These questions will be important as you include more subject areas and more badges and also when thinking about how you can share your project and learning with other teachers/schools.

–       How does the team assure rigour and relevance? 

–       How do you decide what to award badges for?  and who is involved in these decisions? 

–       How, if at all, have assessment processes been adapted to ensure fairness and transparency in awarding the badges? 

–       How are these processes the same/different from business as usual at the school? 

–       What challenges have they encountered so far? (What advice would they have for other schools wanting to start out down this path?).

The second area that we are interested in is around evidence of impact. We touched on this during our Zoom call last week. It would be good to discuss in our next Zoom conversation any further thinking you’ve had around this.

In the meantime, it might be useful to have a look at this website. It has a range of validated survey items for aspects of socio-emotional learning, including self-efficacy. One or more of these scales might be good to use.

Best wishes,

Nina.