A typical module CAIeRO will often start with programme level exercises, such as agreeing or reviewing the mission, ‘look and feel’, and learning outcomes for the programme as a whole. It’s in the interests of the teaching team – and the students – to (re-)use these elements of the programme’s blueprint when designing at module level – it helps to promote coherence and consistency, and to minimise unintentional duplication. But focusing on the module level, while necessary for planning and supporting good learning, can sometimes lead to a fragmented approach over time. Sometimes you need to take a step back to see the bigger picture.
The Learning Design team are often asked to support teams who need to review and make changes at the programme level. As with module CAIeROs, the reasons for this can be many and varied: maybe it’s a new programme or pathway; maybe there have been significant changes in staffing or in the subject area; maybe the team want to respond to specific institutional agendas or to challenges identified in student feedback or grades; maybe it’s just been a long time since the programme was reviewed as a whole and the team want to ensure that iterative changes at the module level haven’t affected the coherence of the award map. Programme level CAIeROs can be as diverse and bespoke as module level ones, but there are some common goals, and as a team we are always refining our toolkit to support them. You may find you use some of the steps below more or less depending on your needs, but this post, along with the Programme Planner developed by our very own Rob Farmer, will give you an idea of some of the approaches available to you.
Before you start: (Re)establishing consensus
View the full Programme CAIeRO Planner
The programme ‘blueprint’
As always we suggest you start by defining the intended outcomes, if you don’t have these already. Although programme outcomes are slightly different to module outcomes, because they are not directly assessed, many of the same principles still apply. They need to be aligned with national quality standards (e.g. the FHEQ and Subject Benchmark Statements), and with any relevant Professional, Statutory and Regulatory Body (PSRB) requirements if the programme is accredited – and of course they have to make sense to non-experts, including students and employers. Once you have your programme outcomes, you can start thinking about breaking them down into chunks of learning that will form your taught modules – or, for existing programmes, reviewing how well they map to the modules you already have in place. Do you need to add, remove, combine or split anything?
Sequencing learning
Taking a programme level perspective allows you to plan the development of understanding and skills across a larger timeframe, to make sure that the scaffold is sound and your students aren’t missing a foundation piece when they reach the higher levels. To get the sequence right, you can storyboard your programme using a range of tools, from paper and post-its to Powerpoint or Popplet. Move the pieces around until the order seems logical, and think about whether each piece needs to be ‘short and fat’ (intensive) or ‘long and thin’. At this stage you might want to consider the placement of elements that are more complicated to schedule, like placements and trips, as well as those over which you have no control, like holidays and closed days.
Mapping assessment and feedback
It can be really useful to overlay your assessments on to your programme storyboard, to give you an idea of the mix of assessment activity and also to identify any deadline and marking ‘bottlenecks’. Programme leaders will usually collate details of the summative assessments across a programme, because this information is required for processes like validation, but we would encourage you to do this for your formative assessment opportunities too. This will allow you to easily see the turnaround time and where students will receive their feedback – and to identify whether it is timely enough to be useful for the next summative task! If you don’t know yet how your modules will be assessed, make a note to come back to this step, or keep a ‘work in progress’ version that you can update with more detail as you go on.
Key skills and ChANGE skills
Find out more about the ChANGE skills
Programme design can be a complex business – and this is before you even get in to the details of the individual modules! We recommend that teams who have a lot to do at the programme level leave at least a full day for this type of work, before moving on to the module level CAIeROs.
This is one in a series of posts about the CAIeRO process. To see the full list, go the original post: De-mystifying the CAIeRO.
Need a CAIeRO? Email the Learning Design team at LD@northampton.ac.uk.
In 2010, Roshni Khatri presented on students perceptions of online audio and online text based feedback. Her experiences on feedback were also further disseminated in the Submission and Grading Electronically (SaGE) working group on 17th December, 2014 as they were felt to be still relevant to current cohorts.
Roshni indicated that the majority of students preferred the audio feedback when compared to the online feedback.
Students commented that:
- ‘The online written feedback was visual and suits my learning style.’
- ‘I heard my name and I paid attention!’
- ‘really helpful for me as I am able to go over the proposal and know which bits you are explaining about. I find this more helpful than having written feedback.’
The presentation noted benefits and problems with the process and made suggestions for further work.
Although we suggest using the Media Gallery on NILE for students to submit work to, there are a couple of drawbacks. You would need to add feedback and marks through a Grade Centre column and students cannot see their submitted work in the media gallery, which has led to a few anxious calls to the LearnTech helpdesk.
The alternative is to use a standard Blackboard Assignment (not Turnitin) and allow students to use the Kaltura Mashup tool – which is part of the text box editor. This creates a Grade Centre column and students will have more confidence that they have submitted their work.
It does require clear guidance though so we have created a basic explanation (with screenshots that will work in any module), that you can paste into the description text box (use the HTML editor) to get you started.
You may be having trouble getting Kaltura screen recorder to work. It turns out that with the latest version of Java as a security feature, applets get blocked if their HTML source and jar file are on different servers. To overcome this, you need to add an exception for both https://nile.northampton.ac.uk/ and https://cdnsecakmi.kaltura.com – this does do the trick! The link below explains it in detail: http://www.java.com/en/download/help/java_blocked.xml (Thank you to Dr.Hendrix, for tracking down this solution).
How does it work on a mac? It appears to VERY picky about browser, OS and java versions. We got it to work in Safari on OSX v10.9.3 with Java 7 – running the NILE site in “Unsafe mode” by going to Safari Preferences > Security > Manage Website Settings > Select Java from the left and selecting ‘Run in unsafe mode’ for nile.northampton.ac.uk. Not ideal, but until Kaltura and the latest version of Java are in synch, it is at least possible to get it to work.
Please contact IT Services (rather than LearnTech) on ext. 3333, if you need to get this working on a uni machine.
Images and interactivity encourage students to explore content more willingly that a list of links and anything that can facilitate students to generate and share content themselves outside of the rather clunky discussion boards and blogs within NILE is always welcome.
thinglink is a free picture annotation tool website that allows content to be linked to external resources and then be embedded in your NILE site or blog. There are also editors available as IOS or Android apps.
Possible uses are as a ‘visual portal’ for students to access further information and resources or – if the thinglink is set to public – as a shared activity where students can add links and annotations to a picture as a class or as part of a group.
You can find more details and try using example thinglinks in this NILEX review.
Thanks to Belinda Green for spotting the usefulness of this app in Education and sharing.
We’re well into assignment season and support calls are beginning to increase in relation to missing assignment submission points, incorrect settings preventing students from uploading one or more versions, apparently missing assignment submission in Grade Centre, grades being released too early (or never) and scores instead of letter grades being given to students.
While it might be tempting to blame the computer*, it’s usually just being a little rusty that’s at the root of the problem. Please take a few minutes to remind yourself of the simple things to do before and after marking to avoid problems by looking at the SaGE survival guide.
NILE isn’t without its problems, but we must be honest with students if we are to be able to interpret their feedback on this critical part of their student experience.
* and posting this in an announcement!
Great learning design and quality assurance are two sides of the same coin. When you make changes to a module or programme, there are a range of QA procedures that might apply, and it helps to understand how these processes interlink – particularly when planning ahead.
New programmes or modules
When writing a new module or programme, a CAIeRO workshop can help you with everything from writing the learning outcomes, to choosing the assessment and creating the learning activities. The first day of the workshop will help you set the foundations, and the second day helps ensure these are workable in practice.
New programmes and modules are subject to validation, and as part of this process you will be required to write a rationale for the new offering. The planning work you do in the CAIeRO can help you complete this, as it includes consideration of strategic goals, the student experience, and resource and training requirements. The CAIeRO process can also help you firm up your curriculum documentation (also required for validation), and make decisions about the allocation of teaching and learning hours, assessment strategy and more. In addition to documentation, the CAIeRO can help you get started creating sample teaching materials, which are required for validation of distance learning programmes.
At the University, many validations can be completed online, but some (particularly those involving PSRBs) require a validation event. For these programmes, validations usually take place in the Spring term. For a new programme starting in September, you should aim for validation early in the term to allow the maximum possible time for development of the course materials (if you want the course to start the following January, you might aim for validation later, in March or April). This means you should be scheduling your CAIeRO in the previous Autumn term.
For more on the validation process, see the validation page of the website, which has information and links to the handbook..
Periodic subject reviews
PSRs are a chance to reflect on what has worked well in your teaching over the previous five years. It is also a great opportunity to use those reflections to define the future direction of the programmes involved. The CAIeRO process can be used to support this process in a number of ways: as a ‘health check’ or review of a programme; to target specific issues you may have identified; and/or to plan how to implement changes you’d like to make.
For PSR you will be required to submit a Self-Evaluation Document (SED). This document will ask you to reflect on things like alignment with frameworks and standards, the currency of the curriculum and student achievement and feedback. All of these elements can be considered within the CAIeRO process, to help you complete the SED form and prepare for any questions during the PSR event.
At the University, PSRs usually take place in the Autumn term, and the documentation is submitted in advance. CAIeROs for PSR can be scheduled at any point in the year (although you may want to note the Change of Approval guidance below when considering timing).
For more on the PSR process, see the PSR page on the website, which has information and links to the handbooks.
Ongoing review of delivery
Of course, adjusting and adapting your teaching and assessment practice happens all year round, and is not dependent on big events like those listed above. You might have taken over a module or programme, or be considering a more blended approach, or just want to try a new idea you’ve heard about. You can book a CAIeRO for issues like this at any time in the year, but you should be conscious of timing the implementation of these changes, and the possible impact on the student experience.
Wherever possible, you should avoid making big changes that will affect current delivery of a module or programme part way through. In addition to this, for level 5 and 6 modules, be aware that students need to know what to expect when they make their module choices. Any changes made after students have chosen the module should be made in consultation with those students.
Changes to existing modules and programmes are achieved through the Change of Approval process, which recognises three levels of change (based on degree of impact). Type B and C changes (more substantial than Type A) must be submitted well in advance of the proposed delivery, and for levels 5 and 6, in advance of the publication of module information to students. For 2015/16 delivery, the deadline for change of approvals for these modules is 13 January (for levels 4 and 7, the deadline is May).
For more on this process, see the Change of Approval page on the website, which has information and links to the handbooks.
Learntech have been examining the world of free online polling tools this week in response to a number of requests. While there are several alternatives with different strengths an weaknesses, Polldaddy stands out as a flexible tool to use in a variety of situations – as a brief informal piece of formative assessment, a survey/poll or just feedback on a piece of content or activity. These items embed very well within NILE. There is a fuller description of Polldaddy on the NILE External Tools blog.
It requires an email registration on the WordPress.com site, with which it is also tightly integrated.
Although Staff have access to Panopto to record their presentations, students have had to do things like embedding audio into PowerPoint to create multimedia presentations until now.
SlidesLive can work in a very similar way to Panopto – download the recording software (for Mac and Windows), record the presentation, upload it and make some editing changes. It is slightly different in that it converts each slide to an image, the advantage being that it’s possible to replace or remove a slide using the online editor. It’s also possible to add a second ‘stream’ by using a video clip from YouTube that can be viewed alongside the slide images – the viewer can adjust the relative sizes of those two pictures.
There’s nothing to stop you adding a YouTube video directly in the online editor, then adding a number of images (these could be captions or even a transcript) which you can synchronise to the video. You’re not obliged to just capture PowerPoint, the recording of the slides is just a screen capture, so you can use other presentation software or just capture a screen (just one screen if you have two).
In its simplest form, most students would have very little difficulty in creating a basic presentation. The down side is that although uploaded presentations can be made unlisted, they cannot be downloaded so are unsuitable for summative assessments. However, presentations can be made public, perhaps as part of a portfolio or an online CV. The slide/video combination also opens up the possibility of more creative uses – critiquing someone’s video presentation or interpreting the content. 
You can find a video/slides example here. SlidesLive does require users to register using a simple email – as ever, we’d suggest avoiding the Facebook login option.
In 2003, Richard Winter wrote a piece for the Guardian in which he listed some problems than can occur when essays are the primary tool used to assess what, and how much, students have learned. Winter claimed that the essays students write often show evidence only of surface learning, rather than deep learning, and that the use of essays for assessment was partly to blame for this. Personally speaking, I really like the essay. I find it stimulating, challenging, and an opportunity to examine and learn in great detail and depth. However, regardless of how much one might like the essay, sometimes it could be useful to provide students with a friendly way in to academic writing, especially for first-year undergraduates or those returning to education after a break, and this is where patchwork text assessment comes in.
The basic idea is that rather than choosing one or two major points of assessment, the students submit multiple short pieces of writing on a regular basis (perhaps in the form of an academic blog or journal) which are then ‘stitched together’ with a final summary, evaluation or reflective commentary. The learning outcomes remain the same, the word count is not altered, and the final submission deadline stays as it is: the difference is that the writing is built up over a longer period of time, in which the students complete many short, directed writing tasks. The writing tasks could be similar or varied, and could be shared for peer-review or kept private. Tutors could review the writing tasks at one or two specific intervals in order to provide formative feedback, or could simply mark everything at the end of the process.
Dr. Craig Staff (Senior Lecturer in Fine Art) and Rob Farmer (Learning Designer) are currently conducting research into patchwork text assessment with students in the School of The Arts. The purpose of the research is to study the attitudes of students to this type of assessment, to look at the quality of the writing they produce, and to determine the staff workload implications when assessing the patchwork texts. The project is due to run over three academic years (2013/14, 2014/15, 2015/16), and will involve around 300 students from levels four to six. Interim findings from the students who undertook the patchwork assessments in 2013/14 clearly showed that it would be worthwhile continuing with the project.
Find out more …
Read Richard Winter’s 2003 Guardian article ‘Alternative to the Essay’
Visit the patchwork text pages of Richard Winter’s website
Visit the Digitally-enhanced Patchwork Text Assessment (DePTA) JISC project website
