Currently viewing the tag: "Assessment Design"
In the autumn 2012, a local General Hospital approached its local Higher Education Institution (HEI), the University of Northampton (UoN), to propose that pre-registration nursing students become involved in working with its Audit Department. Following extensive discussions and negotiations, it was jointly agreed to ‘pilot’ an opportunity for two third year adult student nurses to work on a pressure ulcer care audit as part of their third year dissertation. Supported by members of the Audit Department at the local hospital, and supervised by academic tutors with experience in primary research data collection techniques, the specially selected students embarked on a journey which was to revolutionise their own views of audit, as well as demonstrate to both the audit team and academic team the need to develop their own experiences of supporting student nurses who elect to undertake audit work.
Throughout the duration of the ‘pilot’ project, all parties involved in the process received extensive support from within both organisations. The students were supported in the ‘hub’ placement learning opportunity by their practice assessor, who had been extensively briefed by the Module Leader (ML) prior to the students’ arrival in their designated clinical area. The practice assessors were also in frequent communication with their University Clinical link person throughout the duration of the project. These Academic Links worked with the Practice Assessors to facilitate changes to the students’ working shift patterns as required to support the audit process. In addition, regular local meetings were undertaken between the ML and members of the audit department, clinical supervisors, to address concerns and queries as they arose and clarify both the communication mechanisms for the pilot and the academic requirements of the dissertation.
On completion of the project, the students were formerly invited to evaluate their unique experience via standard academic reporting mechanisms; and informally invited to meet with the Head of Nursing Developments (HoND) and ML to provide more detailed verbal feedback on the strengths and challenges of the pilot. Having completed the first phase of the audit pilot, the ML and HoND reviewed the project with the aim of identifying both the positive aspects and potential areas for improvement should the project be extended into a second phase with multiple-sites. After successfully completing their undergraduate dissertation and achieving their award, the student participants who participated in the pilot expressed both their satisfaction and enjoyment in being given the opportunity.
For more information about this assessment, please contact Dr Jacqueline Parkes, Associate Professor/Head of Nursing Developments (Jackie.Parkes@northampton.ac.uk) or Cindy O’Dell, Senior Lecturer Adult Nursing (Cindy.Odell@northampton.ac.uk)
This case study is taken from the Institute of Learning and Teaching’s 2015 publication ‘Outside the Box Assessment and Feedback Practices’, available from the University’s Assessment and Feedback portal.
At the end of a session ask students to draw a diagram, graphic organiser, model (you do have to explain these at first) to show their learning: what new knowledge, skills and ways of learning they have developed. Share and discuss models.
For more information about this assessment, please contact Elaine Batchelor, Subject Leader—Initial Teaching Training (Elaine.Batchelor@northampton.ac.uk)
This case study is taken from the Institute of Learning and Teaching’s 2015 publication ‘Outside the Box Assessment and Feedback Practices’, available from the University’s Assessment and Feedback portal.
“First year BA Advertising are given a live brief to deliver. Last year it was wrapping a taxi – see article here http://www.northampton.ac.uk/news/advertising-students-
take-part-in-a-taxi-wrap-challenge-to-promote-the-university-of-northampton.
This year they are working on developing a Mascot for the University and using it in a number of promotional situations, such as a Saints game or an awards ceremony. Next year, it will be something different. The main point is that it is real and they have to deliver everything from pitching for funding, through to execution.
I refer to the assessment as ‘Reportage’. A group report documents the steps in the process, with an emphasis on the role their team played. Also, this piece of work includes an individual reflection. I’m not sure that it is earth-shattering in terms of process, but it is authentic, very organic (full of challenge and uncertainty) and the student feedback has been very good”.
For more information about this assessment, please contact Kardi Somerfield, Senior Lecturer in Marketing (Kardi.Somerfield@northampton.ac.uk)
This case study is taken from the Institute of Learning and Teaching’s 2015 publication ‘Outside the Box Assessment and Feedback Practices’, available from the University’s Assessment and Feedback portal.
The BA in Policing programme uses an extended role play involving a robbery incident. This assessment requires students to deal with an incident from first report through to appearing in court to give evidence. The exercise requires them to devise their own investigative strategy and if they make the right enquiries, they are fed extra snippets of information that should lead to an arrest of the suspect. Students interview the suspect using the tape-recorded interview method and then submit the file to the Crown Prosecution Service for a decision as to whether or not to charge the suspect.
For more information about this assessment, please contact Bob Lyman, Senior Lecturer in Criminal Justice Practice (Bob.Lyman@northampton.ac.uk)
This case study is taken from the Institute of Learning and Teaching’s 2015 publication ‘Outside the Box Assessment and Feedback Practices’, available from the University’s Assessment and Feedback portal.
Our colleagues at CfAP are often on the receiving end of poor assessment design. In this post, Kate Coulson, Head of CfAP, describes the impact on the student experience and looks at ways to ensure this won’t happen to your students…
“Maya* is an undergraduate in the second year of her degree. Throughout her first year, she was averaging a C grade in her assessments. Maya has just received her grade and feedback from her first assessment of her second year. She was given a D grade and the marking tutor advised her to visit CfAP to get some support and guidance in “understanding the question”.
When Maya meets with a CfAP Tutor she becomes very distressed and states that “the question didn’t make sense” and “I don’t know what I needed to include”. She also states that she spoke to her tutor directly as she was unclear about the assessment but their conversation left her more confused. When chatting to her course mates about the assessment, they had interpreted the requirements in a totally different way and she panicked and didn’t know what to do.”
When writing questions for essays or assignments it is imperative that you think about the student. Badly written essay questions confuse the student and can affect their confidence and performance in the task – sometimes even leading them to question whether University is the right place for them.
Tips to help you avoid the pitfalls:
- Allow time to plan your questions or tasks.
- Be clear about what knowledge and skills you want the students to demonstrate (these should be informed by your learning outcomes).
- When you are writing a question or task, consider the stage of the programme and module where it takes place, and evaluate whether the students have the content knowledge and the skills necessary to respond adequately.
- When scheduling, be aware of other assessments students will be given from other modules on the programme. Nobody benefits from students having to divide their time and energy between multiple deadlines.
- Discuss the assessment with your students – both the task itself and the purpose of it. Explaining why you have chosen this task, and how it will help them to reach the learning outcomes, will help them feel ownership. Be prepared to adjust in response to valid feedback.
- Share grading criteria and rubrics ahead of the assessment. Students should know what they are aiming for, and what satisfactory performance looks like. Better still, consider writing a model answer. This will help you to reflect on the clarity of the essay question, even if you choose not to share it with the students until after the deadline. It could also serve to inform the grading of students’ responses.
- Use your colleagues to critically review the question or task, the model answer and the intended learning outcomes for alignment.
- Use formative tasks to help students to develop their understanding of expectations and standards. Better still, plan out the ‘assessment journey’ when planning your module, to ensure students have opportunities to learn the process as well as the content of the assessment. The Assessment and Feedback cards from the JISC Viewpoints project can help you do this.
Writing good essay questions is a process that requires time and practice. Review your questions after the students have completed them, think about how the questions have been interpreted. Studying the student responses can help evaluate students’ understanding and the effectiveness of the question for next time.
Useful reading and resources:
The University’s Assessment and Feedback Portal provides more information about assessment design, including links to published research in this area.
The Assessment Brief Design project from Oxford Brookes gives detailed guidance on writing clear and targeted briefs.
For more on the great work done by the Centre for Achievement and Performance, visit the CfAP tab on NILE.
*”Maya” is a fictional character, although her story is based on real events.
Dr Estelle Tarry outlines the development of the URICA Undergraduate Journal, which publishes selected undergraduate dissertations from the Faculty of Education & Humanities.
View the full case study here (PDF, 542KB)
Visit the journal website here: http://bit.ly/URICA
Karl Flowers, Senior Lecturer, Institute for Creative Leather Technology assesses his students through presentations and has often found that it is difficult to schedule face-to-face time slots where all the required parties can make it. Allowing his students to independently record their presentations and submit them through NILE’s video streaming tool, Kaltura, overcomes scheduling issues, enables more use of technology for the students and enables sharing good practice between year groups.
The first stage of a CAIeRO is all about outlining a vision for the programme or module – what is it for? what do you want it to do? – and then drafting this into something concrete to work on.
For new programmes, refining the vision or idea is usually the logical place to start; defining the goals and the parameters, in discussion with colleagues on the course team, and with support from learning designers.
For existing programmes, this step may initially seem quite straightforward. A quick reference to the programme specification should tell us what we need to know. In practice though, programmes and modules get tweaked and adjusted over time, and for many programme teams it is rare to get the opportunity to spend time sharing ideas about the bigger picture of what should be taught, and how students might learn it. Stage 1 of the CAIeRO is your opportunity to revisit the aims of the programme and do a health check: asking is this still what we want to achieve? And if it is, then is this still clearly reflected in the modules that make up the programme?
As with all of the CAIeRO elements, you may spend more or less time on each task depending on the needs of your programme. That said though, it is always worth taking the time to ensure a shared vision before moving forward.
The mission statement
This first activity asks you to define, as a group and in a limited number of words, the ‘mission’ of the programme. This helps to ensure that everyone in the team is aiming for the same thing, so that we design a consistent experience for the learner. This step is particularly useful for programmes that combine modules from different areas of the discipline, and it allows those leading different modules to share their perspectives and experience. Reaching consensus can sometimes be tricky, but it does make everything that comes afterwards much easier! Once you have this, you can create statements for each of your modules, that align with and develop the mission for your programme.
The ‘look and feel’
This stage of the workshop uses Course Features cards, designed by the Open University Learning Design Initiative. You’ll be asked to narrow down the features to those which are most important to the ‘look and feel’ of your course. This may sound a little ‘woolly’, but it’s an easy umbrella term for the different elements that need to be considered (pedagogic approach, guidance and support, content, interaction and so on).
As with the mission statement, this activity helps the team to work towards a consensus on the type of learning experience you want to create. But there are also other gains to this process, that sometimes go unnoticed:
- It provides a common language to help you and your colleagues talk about how you like to teach – particularly for those teaching strategies that are based on tacit experience. Choosing these stimulates discussion about them: what do you mean by …? how does that work? why is that the best approach? This discussion is useful for skill sharing and personal development, as well as narrowing down the most effective approaches for the context.
- It brings the learners into the heart of the conversation, as choices need to be made about what learning approaches they might use, and what kinds of support they might need.
- It helps to ensure that you are considering all the elements that make up a balanced course.
Constructive alignment and backwards design
The next three sections or tasks ask you to focus in on the module level, though you will need to keep the programme-level outcomes and assessment map in the back of your mind to ensure alignment. We’ll look at the building blocks that form the basis of each module: the learning outcomes, assessment tasks and learning activities. We’ll be looking to flesh out the initial vision into a more structured pathway that is constructively aligned, asking: how do we define the learning in terms of demonstrable outcomes? how do we design assessment opportunities that allow the learner to demonstrate achievement of those outcomes? how do we create learning activities that support the learner to reach the intended outcomes and succeed in the assessment? For this we often use a ‘backwards design‘ approach, beginning with what we want the learner to know and be able to do at the end of the module, and working backwards.
First in this sequence are the outcomes (what the student should ‘come out with’, or should know or be able to do). These are arguably the most important element of a module, and not (only) because they are required for quality assurance and benchmarking! Outcomes define the parameters of what will be covered, and help the student to understand what’s expected and what will be assessed. We will check the outcomes for each module against three key criteria: language, academic level, and relation to assessment.
Assessment activities will be chosen or reviewed to ensure validity: what’s the best way for a student to demonstrate these outcomes? We’ll also consider how to prepare the students for the assessment (in terms of process as well as content), and how to incorporate peer and self-assessment.
Finally, we’ll begin to consider what kind of learning and teaching will be needed to support the students in achieving the outcomes. We’ll cover this in more depth in the next section.
A note on paperwork: Any design change to a module needs to work within the relevant quality assurance framework. If you’re working on learning outcomes or assessments, these may already be written in your module specification, and making changes to these could require a change of approval. When designing learning activities, you’ll also need to consider the allocation of teaching, learning and assessment hours that currently make up the workload for that module (200 hours for a 20 credit module). Don’t let QA requirements stop you improving things – your Learning Designer or Embedded Quality Officer can help you to understand the requirements for any suggested changes.
This is one in a series of posts about the CAIeRO process. To see the full list, go the original post: De-mystifying the CAIeRO.
Need a CAIeRO? Email the Learning Design team at LD@northampton.ac.uk.
In 2003, Richard Winter wrote a piece for the Guardian in which he listed some problems than can occur when essays are the primary tool used to assess what, and how much, students have learned. Winter claimed that the essays students write often show evidence only of surface learning, rather than deep learning, and that the use of essays for assessment was partly to blame for this. Personally speaking, I really like the essay. I find it stimulating, challenging, and an opportunity to examine and learn in great detail and depth. However, regardless of how much one might like the essay, sometimes it could be useful to provide students with a friendly way in to academic writing, especially for first-year undergraduates or those returning to education after a break, and this is where patchwork text assessment comes in.
The basic idea is that rather than choosing one or two major points of assessment, the students submit multiple short pieces of writing on a regular basis (perhaps in the form of an academic blog or journal) which are then ‘stitched together’ with a final summary, evaluation or reflective commentary. The learning outcomes remain the same, the word count is not altered, and the final submission deadline stays as it is: the difference is that the writing is built up over a longer period of time, in which the students complete many short, directed writing tasks. The writing tasks could be similar or varied, and could be shared for peer-review or kept private. Tutors could review the writing tasks at one or two specific intervals in order to provide formative feedback, or could simply mark everything at the end of the process.
Dr. Craig Staff (Senior Lecturer in Fine Art) and Rob Farmer (Learning Designer) are currently conducting research into patchwork text assessment with students in the School of The Arts. The purpose of the research is to study the attitudes of students to this type of assessment, to look at the quality of the writing they produce, and to determine the staff workload implications when assessing the patchwork texts. The project is due to run over three academic years (2013/14, 2014/15, 2015/16), and will involve around 300 students from levels four to six. Interim findings from the students who undertook the patchwork assessments in 2013/14 clearly showed that it would be worthwhile continuing with the project.
Find out more …
Read Richard Winter’s 2003 Guardian article ‘Alternative to the Essay’
Visit the patchwork text pages of Richard Winter’s website
Visit the Digitally-enhanced Patchwork Text Assessment (DePTA) JISC project website
“Changemaking is simply where someone spots a social problem and does something about it.”
The Changemaker Certificate has been created and implemented by Fr Tim Curtis in the School of Health as a way of encouraging all staff and students across the University to explore what it means to be a Changemaker and also to award engagement with Changemaking through the completion of a Social Venture Canvas (SVC) that would be capable of attracting financial support.
The Certificate sits in two places at present: (1) tied to academic modules offered from the School of Health – SWK1051 and SWK3025; and (2) as an extra-curricular award not tied to academic credit. The route taken will depend on whether or not the academic modules are offered as part of individual programmes, with the idea being that anyone across the University can therefore participate.
Through a series of coursework activities that all feed in to and contribute towards attainment of each section, participants prepare a draft proposal for an SVC that will be both self- and peer-assessed. Work is assessed using a newly-designed ‘CM Assessment Rubric‘ that draws on and modifies a number of identified 21st Century Skills. Achievement comes either through attaining the Certificate in isolation at either bronze, silver or gold level; or, for students on SWK1051, the rubric provides clear links to either the gold, silver or bronze certificates, but uses submission deadlines and the standard UMF scores to determine achievement levels. While it may depend on the social venture being addressed and the work carried out by the student, students completing SWK1051 are most likely to attain the bronze certificate (which is around concepts of thinking and exploring a social venture idea), with some credit at silver. The final venture canvas would still require to be submitted to the Changemaker Certificate system however.
- For more on the Changemaker Certificate case study, please see Changemaker assessment final.
- To enrol on the Changemaker Certificate for yourself, please click here.
