Currently viewing the tag: "Assessment Design"
Student perceptions of their learning and engagement in response to the use of a continuous e-assessment in an undergraduate module.
Naomi Holmes, School of Science and Technology
Dr. Naomi Holmes (School of Science and Technology) undertook the use of low-stakes continuous weekly summative e-assessment with a cohort of level 5 (2nd year) students. Biggs and Tang (2011) state that it is assessment and not the curriculum that determines how and what students learn. Learning needs to be aligned with assessment as much as possible to increase engagement, even if the result is that the student is “learning for the assessment”, and therefore accreditation. With this in mind the use of low-stakes weekly assessments was undertaken to help support learning (formative), and lead to accreditation (summative). Results show that both physical and virtual engagement with this (optional) module, and students’ learning and understanding of the subject increased because of this method of assessment.
Liam Fassam, Lecturer in Operations Management in NBS, has recently been using the Socrative student response system as part of a deliberate effort to increase learner engagement with the subject and provide formative feedback . Liam made a conscious decision to use the beta.socrative.com platform rather than the Socrative app to ensure an easy and quick in-class response. From an initial trial of this software, he has since begun using it on a weekly basis, having found the process so simple and straightforward that the time it takes him to pre-load up to 20 questions for use at the start of a session is down to around 10 minutes.
Initially looking for technology that could be used in the classroom in a way that he believed would be aligned to the demands of modern students, Liam has benefited more widely from this approach as he can obtain weekly analytics that enables him to evaluate student progress and build up a clearer picture over time. Conducting the quiz in groups provides a level of safety for students who may feel unsure as to their understanding or who are uncomfortable with the idea of identifying themselves on an individual basis. This also overcomes any potential accessibility issues within the classes. The group aspect has also given rise to some healthy in-class competition which he likens to a ‘football league’ feeling where groups are vying to be top of the league.
In addition to the MCQ approach, Liam also uses Socrative on an ad-hoc basis to get a feel for student opinion on a related topic. Where a show of hands around a controversial topic might not produce as accurate a response as he might have hoped for, the anonymity of the Socrative approach serves to overcome fears of potential exclusion or isolation on the basis of ‘non-PC’ answers.
So, what’s the catch? Well, at the moment Liam has experienced no technological glitches or difficulties, but that may be because he has only used it on-campus in Northampton where students have access to digital technology and a strong wifi connection. He would like to introduce its use off-campus, engaging his online learners around the world, but is aware that they may well experience bandwidth issues. Another plan is to look at ways of embedding it into NILE.
His advice for those who are interested in trying this out? Just do it! And if you do, be consistent because once you start, your students won’t want you to stop! In short, beta.socrative.com is a really good tool for subject-matter validation and Liam was genuinely surprised at the level of learner engagement and acceptance.
So if you are looking for a quick polling tool that makes use of devices already owned and brought to class by your learners, then why not give this a try?
Last December, Economics Lecturer at NBS, Dr Kevin Deane, took the unusual step of abandoning 4 weeks of his timetabled lecture programme and replacing it with a group exercise culminating in an academic poster exhibition (see this blog posting for more details!)
The exhibition was a real success, particularly for a first-time event, as evidenced by the comments from the students and other NBS staff who attended the exhibition. That said, Kevin has some definite changes and improvements he would introduce next time around. But, in these days of NSS scores and working to improve the student experience, the big question to be answered is … what did the students think?
Generally, their reflections mirrored those of Kevin himself. Apart from an appreciation of the refreshments (!) the following comments are worth mentioning in response to the question of what was good about the task:
- One student responded by saying that the good thing was the “big range of information exchanged and displayed, very insightful. Food was good, getting tutors and guests involved.”
- Another enjoyed the fact that they didn’t have the pressure of an assessed assignment.
- “People did them relatively well. Rewards were good incentive”.
- Another student commented that it “was a great insight into a variety of economists. It provided me with a better understanding of these economists and their philosophies.”
There were some technical hitches on the day of the exhibition itself. In particular, Kevin had been expecting the room to be ready when he and his students arrived, but an error in communication meant that an hour was lost having to set up the exhibition boards. This did have a significant knock-on effect for students as the first hour of the session was lost. This had been scheduled for a student-student presentation of each of the posters, which would have provided the primary opportunity for students to learn about those other economists being studied by their peers.
The following student comments on what could be done differently/better mirror Kevin’s own reflections. Specifically, the students were keen that copies of the other posters were circulated – something Kevin had already planned to do. This is of particular importance pedagogically – where students are creating and generating module content which forms one jigsaw piece of the whole picture, ensuring that each student has access to the full and final picture is essential. Another comment was that it was rightly considered unfair that some students were asked to produce posters on economists that had already been studied in class whereas others were starting from scratch. Looking ahead, Kevin would ensure this didn’t happen again and recognises that it was purely circumstantial, arising from the decision to move away from lecturing to the poster exhibition after the lectures had begun. In itself, this was engendered by student feedback indicating that the lecture approach to this topic was dry and uninteresting.
One final comment worth addressing directly was that students considered the poster to be “too much extra, [it was] not part of our course.” This feedback reflected a failure to appreciate that this poster wasn’t actually ‘extra’ work per se, rather a change in the way the module content was being taught. In future, Kevin would draw attention to the fact that the requirements of producing a poster are no more onerous in terms of the expected study time than indicated in the module spec: 4 students per group x 3 hours per week of independent study x 3 weeks = 36 student study hours per poster.
Other negative comments included the following:
- It was a lot of work, for no obvious reward in terms of assessment.
- Lack of assessment meant no incentive to produce high quality.
- Doing a poster on one subject was limiting.
- Some people didn’t even go.
Having allowed time for both his personal and the students reflections, the following changes would, Kevin believes, improve the exercise next time around:
- Ensuring that the dedicated time for student-student presentations is preserved to ensure that all students receive the benefit of the work done by other groups and learn about all the economists studied in the module
- Explicitly recognise the focus that students place on assessment and grades and therefore turn the task into the first assessment for the module and run it earlier in the academic year when students were not under pressure to complete assessment tasks for other modules
- Ensure that the key points are captured in a summary ‘timeline’ lecture that places them all in context.
Probably the biggest objection he has to overcome is the idea that this task placed an additional burden on students and this really boils down to managing their expectations more explicitly. A clearer explanation and on-going reminder that the poster itself should be the final product of 36 student study hours (9 per student) would go a long way to removing this objection and engendering in students the realisation that this level of work and time investment is, ultimately, what they are at University for!
What do you do when you have a very dry topic to teach and the snores from the lecture theatre are drowning out your words?
Kevin Deane, a new Lecturer in International Development in NBS, faced exactly that problem, some 5-6 weeks into the term. Feedback from the students was clear – “we are bored by this and we are not engaging”. It was time for a radical rethink.
Kevin and I spent about two hours batting ideas back and forth over how to help his class see how the opinions of these long dead men could be relevant to them as 21st century students of economics. I had made a choice not to lecture a group of postgraduate students during a three hour session but instead to spend that face-to-face time on the application rather than the acquisition of knowledge. Could a similar approach be utilised in this context?
The answer came in the form of an academic poster exhibition with each group producing a poster on a different economist which they had to present at an exhibition at the end of term to fellow students and staff within NBS. They also had to explore the relevance of each economist – did they agree with their theories; were their opinions wrong? This had the effect of ensuring a higher level of participation than might otherwise have been the case, given that the poster was not being assessed.
The subsequent 4 weeks of lectures were therefore abandoned and the time allocated to group work on the posters. In spite of some initial discontent Kevin made it clear to the students that this was simply a different approach to teaching and that the students would still be expected to attend and participate. Success was also encouraged by ensuring that students had weekly interim goals and deadlines to work to.
At the exhibition, it was evident that the students I spoke to had engaged with the material and enjoyed finding out about their allocated economist. They had also grasped the concept of what an academic poster was about! A number of staff from NBS were present to ask questions and to help Kevin judge the best poster(s) – three prizes were awarded in the end.
On reflection, Kevin will definitely repeat this approach for this module, but will add an element of assessment to further increase participation and engagement. To read more about the what’s, why’s and wherefore’s, please read his case study – Kevin Deane – Histor.
For now though, this process of resuscitating the wrong opinions of dead men shows that the theories really do live on.
Have you ever wondered if students bother to read the feedback that you so carefully provide them with? Have you ever been overloaded with providing formative feedback that students can use without necessarily engaging more deeply with your advice?
In this case study, Maggie Anderson, Senior Lecturer in Human Resources Management in NBS, reflects on how addressing this issue through the CAIeRO process changed her pedagogical approach to the provision of formative feedback/feedforward and how the Journal tool can be used to encourage earlier student engagement and increase individual learner responsibility.
Kate Williams (Senior Lecturer in Journalism & Broadcast Journalism) has used NILE (Blackboard) rubrics in conjunction with Kaltura videos to allow her students to submit videos through the Assignment tool on NILE, embed the video into their submission, and then marked the videos using a rubric. This allowed Kate to maximise the amount of time she spent feeding back on students’ work and less time worrying about technical issues. Kate also found that by using a set of definitive marking criteria she was more consistent, objective and transparent. Click the link below to view the case study.
Using Blackboard rubrics to grade video assignments
This case study describes using Panopto and iPad for easy and mobile recording of student assessments.
The main aim was to record student assessments and upload immediately to our VLE, minimising editing and publishing while maximising availability for external moderation.
Case Study (case study, PDF 171.6 KB)
Kim Stuart and Leonie Siddons presented on PebblePad for portfolios at the Learntech Conference 30-5-12.
This case study looked at introducing students to Turnitin and how to use the Originality Reports for revision support.
This is an interesting case study on the use of the multiple choice tool in NILE to assess students. Have a read and feel free to comment.
