Context & Scale
This pattern is concerned with providing personalised feedback to students at scale to create connections with students and support their learning and experience. It is suitable for use in any large unit. Using appropriate tools, teachers prepare and trigger feedback to students on their learning and progress at various stages throughout a semester. The relevant feedback is emailed to students based on data of interest from the learning management system. This might include students’ grades, their attendance in face-to-face classes, or the extent to which they have engaged in particular online learning activities such as quizzes. Boud & Dawson (2021) include “using technology to enable more efficient/scaleable feedback processes” among the activities that feedback-literate teachers do (p. 6).
This pattern addresses the challenge of providing quality feedback at scale, because it helps to foster a more relationship-rich environment, supports an efficacy-centred approach, and can lead to better performance and a sense of belonging in a large class. Sending personalised feedback emails in high-enrolment courses that address factors such as student performance, strategies and approaches to help students learn, and the availability of the teacher and other resources to support success, have been linked to higher scores on exams, homework and final course grades (Felton & Lambert, 2020, p. 90).
Problem
The larger a student cohort, the more difficult it becomes to provide meaningful, personalised feedback. Due to time constraints, feedback in large classes is often delivered as marks, with minimal (if any) written feedback. Other strategies teachers use in large classes include giving more generalised feedback to students on assessments based on the performance of the whole cohort. While this may have some benefits for student learning, it is not personalised, and may not be relevant to a proportion of students. The challenge in large classes is maintaining the qualities of effective feedback when teaching a large cohort of students.
Solution
Design a schedule of personalised messages to be sent to students via email at various stages through the semester to provide encouragement and feedback on their progress and learning. Use a tool with mail-merge capacity such as SRES that allows you to compose customised, templated messages, and specify the data of interest and the conditions under which an email will be sent to students. As the system uses student lists, emails automatically include the student’s name at the top of the email. This personalisation helps students feel more connected to their teacher in a large cohort and allows teachers to provide quality feedback at scale informed by data about different aspects of student engagement and performance from the LMS.
Implementation
Consider where feedback will have the most impact on student learning (Boud & Dawson, 2021). Decide what student-related data in the learning management system (LMS) is important. This data will be the basis for determining which pre-prepared feedback will go to which students. For example, if you have recently introduced online modules into your course as pre-work, you may be interested in data on the extent to which students have accessed the course site at a particular stage of the semester. Or if students have a mid-semester exam, stratified feedback can be targeted to each grade level.
Decide how you want to apportion feedback. For example, if the data includes the number of visits to the course in the LMS, you might prepare three different emails. You also need to decide on the criteria related to the data which will determine which email is triggered. For example, decide what constitutes a low, medium and a high number of visits to the site (e.g. between 0-5, between 6-50, and >50). This may depend on factors such as how many weeks the course has been running for.
Create a feedback release schedule to clearly map out when you would like the various emails to go out to students to best support their learning. This can be done using a simple Excel spreadsheet showing which week of the semester the email will go out, the purpose of the email, the associated feedback options, and the data criteria attached to each. Sequencing feedback events can help to maximise their influence on student learning (Boud & Dawson, 2021).
Consider sending an initial email personalised with each student’s name to your whole large cohort at the beginning of the semester. Following an example cited in (Felton & Lambert, 2020), the welcome email might explain something about your background, your expectations of your students, and an affirmation that you are confident that the student can succeed in the course if they “work purposefully and persist through struggle” (p. 88).
Ensure that your feedback encourages positive motivation and self-esteem (Nicol & MacFarlane-Dick, 2006). This is particularly important in the large class context where students may not have as much direct access to their teachers. Where students are receiving an email based on performance that is below what is expected, include a clear description in the feedback of what good performance looks like, and how they can go about closing the gap between current and expected performance (Nicol & MacFarlane-Dick, 2006).
Ensure that feedback is as rich for high-performing students as low performing students. Feedback for high-performing students can reinforce what they did well, and provide additional resources to extend their learning.
While some learning management systems allow you to send emails based on particular criteria associated with data collected through the system, the type of data may be limited. Using a tool such as the Student Relationship Engagement System (SRES) provides more flexibility, allowing many more options for tailoring feedback based on data of interest.
Examples of pattern in use
Undergraduate unit in Business Analytics
This pattern was tested in a first-year undergraduate core unit (Quantitative Business Analysis) in the Business Analytics discipline with approximately 860 students.
The aim was to help students to feel like they had a personal connection with their teacher in this large cohort, and to:
- provide meaningful feedback to students about their progress and performance;
- provide strategies and approaches to help them learn;
- and emphasise the availability of tutors and other resources to support success.
The approach was iteratively developed and implemented. We acknowledge the unit coordinator Bern Conlon and the team at The University of Sydney behind the SRES system.
Description
This unit utilises a flipped approach to learning, where students complete an online module each week, followed by a synchronous workshop where they apply their learning to real business analytics problems. The purpose of each of the five emails sent out during the semester is summarised below.
Email 1: Completion of the online modules in the unit is considered to be vital, as it introduces students to the key statistical methods they need to learn, and sets them up for practicing their skills in the workshops. For this reason, it was considered important to communicate with students in a personalised way about their engagement in the modules. Therefore, the first email sent to students early in semester was based on data from the LMS on the extent to which students were engaging. The image below shows the email variations and the number of students who received each email based on data connected to the completion of modules:
Email 2: The second email was designed to provide feedback and guidance to students based on their performance on the homework quizzes in the first five weeks of the unit and help prepare them for the first individual assignment. Here is an excerpt from the email to students who scored low on the quizzes:
Dear [student’s name auto-filled], well done on completing the first half of BUSS1020. I thought this was a good time to reach out again and provide some feedback based on your level of engagement and progress with the weekly homework. Your homework scores suggest you may be having some challenges applying some of the concepts and methods we have covered in the lectures and tutorials. It may help for you to go back and rewatch some of the core concept videos in any modules in Canvas you are finding challenging. I would also encourage you to redo the practice questions on the Canvas pages to check your understanding. This may help you to target the areas you are finding more difficult and help you with your first Assignment. We do find that when students keep up with the weekly homework and focus on maximising their results, they tend to perform better overall in this challenging unit. Please also reach out to your tutor… etc.
Email 3: This email provided feedback and guidance to students on their performance in the first assignment.
Email 4: This email was intended to prompt students who had ‘no’ or ‘limited’ access to the online modules in the later part of the course when content became more challenging. Students were sent an email if they had not accessed pages in the unit’s Canvas site that provided important core concept videos and activities.
Email 5: The final email provided feedback to students based on their results in Assignment 2 and included strategies and encouragement associated with their preparation towards the final exam.
Implementation
01 > Work out where emails are likely to have the most impact throughout the semester in relation to key learning activities and milestones, and which data from the LMS would help distinguish the kind of feedback needed to help students succeed.
02 > Map the sequence of emails in an Excel spreadsheet including the week of release, the release date, the purpose of the email, the email text variations, and the data criteria to be used to trigger each email.
03 > Use SRES to set up and send the emails at the specified time.
04 > Take note of how students are responding to the feedback. Where relevant, adjust future emails based on student feedback about how useful they have found your messages.
Technology/resources used
An Excel spreadsheet was used initially to map out the sequence of personalised emails that would be sent to students throughout the semester. It included which week the email would be sent, what the primary purpose of the email was, a draft of the email options, and the criteria linked to LMS data that would be used to ensure the correct email when to students.
The Student Relationship Engagement System (SRES) was then used to set up and send emails to students.
Findings
A number of students responded to the email they were sent and articulated their study plans and how they would rectify their low engagement:
“So, I planned to review chapter 1-3 this weekend and go through all materials and questions carefully. Thank you again for remind me, I feel so warm about this. And I will directly ask for help if I need to.”
“Thank you for reaching out and for the concern! I joined the BUSS1020 unit a few days ago which is why I haven’t been able to complete much yet and still have to do the modules/quizzes. I’d like to apologize for my lack of engagement with the modules, as I am still trying to get hold of the learning materials and trying to balance it with my other units. I really appreciate the feedback and can assure that I’ll be done with 3 weeks of buss1020 content by the weekend! Thank you so much.”
Many students indicated that they appreciated the contact from the coordinator:
“Thank you for reaching out to me.”
There was also evidence that many students enrolled in the Business School’s complementary Peer Assisted Study Scheme due to the link provided in a number of the personalised emails.
While in the majority of cases, the teaching team were confident that students were receiving the appropriate email based on their level of performance and engagement, occasionally, a student would challenge the email they received. For example:
“I assure you that I have not been complacent, as I know that the unit will progress in difficulty in due time. Nevertheless I greatly appreciate your email, and I am pleased to know the care and effort the staff at this university place into their students!”
Postgraduate unit in Finance
This pattern was tested in a second-year postgraduate core unit (Finance: Theory to Application) in the Finance discipline where 1000+ students enrol each semester.
The CLaS co-design project for FINC6001 began in semester 2, 2020. First, a design workshop was held with the teaching team, alumni and students. Students flagged feedback and assessment as their top issues. Providing more personalised feedback to students at scale was paramount.
The approach was iteratively developed and implemented over a period of three semesters. We acknowledge the unit coordinators Danilo Lopomo Beteto, Evelyn Lai, and the team at The University of Sydney behind the SRES system.
Description
In such a large cohort, students don’t have many opportunities to get feedback from the Unit Coordinator. SRES was used to create a schedule of personalised messages to all students based on their online participation and the timing of the semester.
First, emails were sent to all students before census date, based on their module completion rate on Canvas, to ‘nudge’ students towards greater participation, to remind them of support services, and to congratulate them on their efforts so far.
01 > A FINC6001 email address was set up so that the teaching team could share the queries. In meetings, the team became familiar with the technology.
02 > All messages were carefully crafted in a Word document. Dates were scheduled according to key student milestones (census and assessment due dates, for example). The messages were then copied and pasted into the system and tested and refined.
03 > Student responses were evaluated. The processes were documented for FINC6001 teachers.
Technology/resources used
As in the example above, the Student Relationship Engagement System (SRES) was used to configure and send personalised emails to students. However, other systems such as OnTask could be used where available.
In-built functionality in Canvas LMS was also used to message students who had not yet submitted assessment. See How do I send a message to students from the Gradebook?
Findings
Students were pleasantly surprised in the majority of cases, and it encouraged many students to reach out to the Unit Coordinators where they might not have otherwise.
Students also enjoyed being acknowledged for their progress as these comments from a focus group demonstrate:
I think it motivated me to not feel like I’m just learning online. It made it a bit more personal. So I liked that feedback. And then my email was we can see that you’re engaging with the course, that’s really good, good luck for your exams and keep studying the material. And for me that was “oh wow,” that was like someone patting you on the back, it just gives you that little extra that I’m just going to keep doing what I’m doing because it must be the right thing.
It showed me that I’ve got some great support from the teaching team, and that there’s help available if we need it. It caught me off guard but it was great to see that email.
Leave a Reply