How Do We Reframe Assessment? Using Technology for Formative Feedback

Tom Henry:

Hello and welcome to this webinar hosted by EdQuarter in conjunction with GoReact. I’m Tom Henry and I’m today’s event moderator, so today we’re going to be talking about digital transformation, of course assessment within universities. A scenario that’s seen rapid growth in no small part due to the pandemic. Current assessment practice increasingly includes activities that could be turned assessment as learning, i.e., that assessment itself is a vital part of the learning process. In particular, we’ve seen the rise of video as an advantage tool that can be used to record skills and for the giving of feedback. There are many advantages to such systems, but areas for debate too, and in this webinar we’ll be exploring good principles for such assessment. Before I introduce the panel, please let me give a warm welcome to you, our audience. This is a live event of course, so could I ask that you keep your microphones muted throughout.

However, we would really like to hear from you and if anybody wants to introduce themselves, letting us know where they’re from and what their position is, then please you’re more than welcome to use the chat box. The chat box is also there to post any questions or observations you might have, and again, we’d really welcome these as part of our discussion today. If I can get a few questions and I can drop them in towards the end and we’ll hopefully get as many as we can answered.

Okay, so let me introduce the panel. First, we have Graeme Knowles, who’s director of education innovation WMG at Warwick University. Graeme joined WMG after a successful career in manufacturing quality management. He’s designed and delivered learning and development programs across the world at a range of educational levels and in partnership with both international educational institutions and global businesses. He’s a track record of leading innovation in all aspects of learning and teaching and has been recognized by a Warwick Award for teaching excellence.

Next we have Josh Beutler. As Vice President for strategy and business development in GoReact, Josh leads the company’s strategic efforts to humanize skills-based learning through adoption of video assessment software. He works closely with many of the world’s largest academic institutions and e-learning companies to help them reimagine how they engage learners through authentic assessment as a foundation for any learning experience.

Next we have Sarah Knight, so Sarah is the head of learning and teaching transformation within the higher education directorate at Jisc and is supporting universities with a digital transformation of learning, teaching and assessment. She established and runs the Jisc student experience experts group, an active community of practice, which provides valuable consultation and dissemination opportunities for Jisc.

And finally, last but not least, we have Jenny Gordon, who’s director of International Market Development at GoReact. In her role at the company, Jenny leads the strategic and tactical efforts to expand into new markets, bringing skills-based learning through adoption of video feedback and assessment to new users all around the world. She’s worked within all phases of education to successfully implement educational technology solutions intended to improve learning outcomes for 20 years.

So that’s our panel. Let’s start by asking about the ERA pre-pandemic, if we remember that era. May I just first ask the panel, what were the norms as far as student assessment and feedback were concerned and how has this changed in three, two to three years? Perhaps Sarah, I could bring you into this first.

Sarah Knight:

Thank you Tom and delighted to be here with you today. I think from our observation in Jisc, we have seen a considerable shift to the use of digital in supporting assessment and feedback. We know that many universities had to make a pivot very, very quickly to support assessment during the pandemic and hence the implementation of digital assessment systems. For many universities they are continuing on that journey and looking at refining and enhancing the way that digital is supporting assessment, but I think also starting to rethink assessment in the way that technology can support feedback, and very much looking forward to hearing more of the discussions that we’re going to be having today in relation to how technology can support and enhance feedback for students.

I think some of the areas though that we are picking up in terms of some of the challenges with that is that we do need to ensure that our students understand about academic integrity, that they understand the purpose of assessment and that they’re engaging more with feedback during their process. We are seeing a great shift and I’m hoping that we are not going to be going back to more traditional forms of assessment, but recognize that there is a challenge there in moving staff forward to continue that pace of innovation that has been ongoing during the course of the past two years.

Tom Henry:

Okay, thank you Jenny. Graeme, perhaps I could bring you into this, what was your experience of assessment pre-pandemic?

Graeme Knowles:

So yes, pre-pandemic, we were quite traditionally in WMG. In the University of Warwick we had a lot of exams, a lot of written assignments, the written word was king. With a few bits more authentic work going on and presentations, group presentations and so on, but we had to pit… As Sarah said, we pivoted very quickly and with some traumatic consequences, so we had to put 130 different modules online within about four months in WMG, so that’s quite a large change, and we recognized that our assessment practices were not going to work as well in the digital space just like some of our traditional lecturing practices didn’t work as well in digital space. We have reviewed that and as Sarah said, we’ve moved very much more to online based to try to think more about how the feedback can be useful and how the assessment can be authentic, so that’s been a sea change within our approach.

Tom Henry:

And for Jenny and Josh, as video assessment, as part of the video assessment company, was it a difficult task pre-pandemic to persuade universities that the kind of technology we’re talking about today was useful or was it really the pandemic that suddenly brought this in into very, very sharp focus?

Josh Beutler:

Jenny, go ahead. You’re welcome to start.

Jenny Gordon:

Certainly it brought a lot of things into sharp focus for a lot of people. We had one of our busiest ever times at GoReact from March, 2020 throughout that summer. With institutions finding initially actually the scramble was we have to do some sort of assessment. We have to do something. What can we use? What can we potentially plug a gap with that? Because there wasn’t anything in place to do something completely digitally. In other instances, organizations were already well on their journey of considering a longer term solution for digital assessment, but I think what then came out of that was the continued use of seeing how in this case GoReact was used to fill that gap and how that then it could be used as more of a substantive solution later. Although there was a need and a crisis, I do think a lot of institutions were on a journey to considering it anyway.

Tom Henry:

Josh would, you would concur with that. You were seeing-

Josh Beutler:

I think about those days beforehand and now that we look back, we see murmurings of all of the gaps, we see people wondering and questioning. Then we as GoReact as a newer technology we had to do a lot of work around just educating potential users what was possible, because they hadn’t even really occurred to a lot of individuals or programs what was possible, and in that process we found ourselves in the midst of COVID thinking, “Now is the time to make sure that people really understand what’s at their fingertips.” And that was super challenging but also fairly exciting as well to see people come around so quickly to say we have to think differently even though the circumstances were significantly challenging as well, but we saw that early on.

Tom Henry:

In my introduction, I mentioned in passing assessment as learning, but I think it might be quite interesting to define this. Sarah, could I ask you what you understand by this term? I think it is the term you’re pretty familiar with.

Sarah Knight:

Yes, thank you Tom. In Jisc we did a large piece of work earlier this year in consultation with the higher education sector to do a review of the landscape of assessment and feedback. And through consultation with the sector what was coming to the fore was that we were moving from earlier work, which was very much focused around assessment of learning to what was then turned assessment for learning. Where we are now, where we are very much promoting assessment as learning, and crucial to this is the understanding of the part that feedback plays in helping students to monitor and regulate their own learning and trying to ensure that any feedback activity was actually feeding forward and needing to further to future improvements.

Much of the emphasis we’re seeing now in our assessment practice is what we would term assessment as learning. The very active undertaking assessment and feedback activities is an essential part of the learning process, and we have a new guide and some guidance on our website and I will share the link in the chat for delegates to have a look at. Which goes into more detail about the model and also then fed through into the seven principles for good assessment and feedback practice, which really does have that principle, that concept of assessment as learning at its core, but also recognizing that a definition is a way to engage staff and students around what it means to them. I’m sure that that other members of the panel will also have some interesting definitions of what that means in their context as well.

Tom Henry:

Yes, absolutely. Graeme, perhaps you could explain this in the context of the way you work at Warwick?

Graeme Knowles:

Yes. In WMG we’re very much that, my favorite Socrates’ quote, which is education kindling the flame not filling over vessel. I think that if you regard assessment in that regard, assessment should be adding fuel to that, the fire of learning, if you like if I’m not stretching that metaphor too far. I think we focus on a number of different things. One is that we think assessment should be integrative, so it should be asking you to draw together various aspects that you’ve been exposed to so far in your learning journey. We also feel it should really promote deeper reflection both about the material that you are being assessed on but also about how you approach that assessment, how you approach learning, and a keyword, resources or authenticity. A lot of our program or we have many programs which are degree apprenticeships where the students or the apprentices are employed and living in a real world of work. We’re very keen that the way we assess reflects something useful in the future professional lives of our students.

Tom Henry:

And for Josh, what’s your understanding of this term and how have you seen it develop within what you do?

Josh Beutler:

Well, Graeme and Sarah both just sparked something in my mind that it’s fairly new, but I wanted to share it just from a realization. I think anyone who’s gone through a university or been in a program and reaches the moment where they know they’ll be assessed has a bit of a emotional reaction. There’s a bit of an anticipation, a little anxiety that preparation for saying, “Now this is where it matters.” And I think assessment as learning takes that anxiety that’s a precursor or something early on in the process and should bring it into this thought process of, “I’m going to do this but it’s going to be for my ongoing growth.” And the anxiety level, the anticipation steps into much more of a, that flame that Graeme just mentioned.

It’s saying, “Here’s what I know and when I interact with someone else or when I get someone else to show me something differently, I’m going to learn something I may have never seen or never understood until I see myself doing it with the right connection with someone else.” And so a removal or an anticipation of a shift from the anxious assessment into a much more strength oriented ownership of learning is something I would say, is something I’ve been thinking about from what the difference is from between assessment as versus assessment for.

Tom Henry:

Okay, thank you. I think on the back of that, perhaps I should ask what the advantages of a video based assessment system are. Obviously apart from the obvious that we don’t have to be in the room at the same time, et cetera, but what does the panel see as the really key elements in this? Jenny, might I ask you first?

Jenny Gordon:

I think the video solution is a solution that solves many challenges, those that have come up recently and those that are going to continue being difficult, the travel, the cost of travel, the way of getting multiple different stakeholders to assess a student or a learner in different places, all of those things that come up regularly, but then there’s added value as well. Once a piece of content has been created on video that can be reviewed by yourself, you can self reflect on that, not only once but many times, and the ability to then share that with the people that you are getting formative feedback from time and time again, so I think it becomes a resource that can be used multiple times in one of our key disciplines is teacher training.

A lot of our trainees will say, I used to have somebody come into my classroom and give me my feedback on my piece of paper, which was great, but now I can see that feedback in complete context with what I was doing, and that combination of pinpoint feedback next to the performance of a skill or the performance of a specific competency allowed them to then implement the change or that feedback much more specifically than it had previously when it was detailed on perhaps a form or a feedback database. I think there’s lots of different advantages depending particularly in which to discipline or thing you are teaching, but I think as Graeme said, the more vocational requirement, the deliberate practice of something and then the need to deliberately assess at the end of a process, video can be very supportive in that, in bringing stakeholders from all over to look at something that’s happening either live or after the fact.

Tom Henry:

Sarah, what’s your view on it?

Sarah Knight:

It’s about the authenticity, the experience and that’s something that has come through our research is that it really is important for students to have the real world experience. Healthcare profession is one example where that lends itself really well. Obviously you’ve mentioned Jenny, through teacher education, but really ensuring that students are assessing the skills and knowledge that is required when they move out into the workplace, so I think that’s the one aspect.

I think the other aspect is that personalization of the feedback and students really do value that personalization of feedback and I think that’s another huge advantage, but also I think the timeliness of it as well, it’s giving that meaningful feedback at the right point in the right context and that really does help students to engage better with feedback.

We know obviously from our research that we’ve done in Jisc, but also from the recent report that was published by the advanced HEPI around student experience survey, that feedback is always something that students respond quite negatively to in terms of quality. The more we can enhance the timeliness of feedback, the specific feedback to help students improve rather than generic comments, that is all going to support that assessment as learning and to really help students feel that they are engaging and having a better understanding of what’s being assessed.

Tom Henry:

And Graeme, do you have any examples of best practice from work with your work at Warwick?

Graeme Knowles:

There’s a number of different things we’ve done. One is, and this relates to healthcare environment. We have a digital healthcare scientist program and they do as part of that program, they do what they call OSCEs, which are objective structured clinical examinations. Which are practical examinations where they’re faced with the patient and they deal with that patient, and they get feedback on that and obviously it’s part of their degree, but one of the important things is that they now get videoed and they also get feedback on the video that they can look at in a later date, because obviously taking feedback when you are being assessed is quite a stressful thing. You’re not really focused on the feedback so much as am I going to pass? And having that opportunity to revisit and reflect on it with their tutors going forward, so discussing it with their tutors as well, that has really helped them to develop their skills.

The other thing we’ve done, which has been a little more convoluted but I think has been very useful is we had one example where we had some students create a video as part of their, as a team as part of a module, and those students were then asked to comment on anybody else’s videos and then after having done that, they were then asked to re-comment on their own videos, reevaluate their own videos in the light of what they’d seen in other people’s videos and the problems and issues they’d seen with those as well as the feedback they’d had from their colleagues. It got all very meta, but was able to enable them to really reflect deeply about what they were doing and what they were learning, and obviously we can easily see problems in somebody else’s work that we maybe don’t spot as easily in our own work, so it’s really, they’re learning a lot from that.And we’re getting huge, very powerful figures back from people’s that 90% of the people saying it deepens their learning. Again over 90% it helped them to feel, to understand how their group was progressing and to learn going forward. I believe in this is a really powerful way of underpinning reflection for our students, which is for me an absolute meta skill for all students helping them to reflect, giving them a chance to reflect in the moment but also reflect at leisure. Sorry that was a long answer but-

Tom Henry:

No, no it’s an interesting point as well, and as you were explaining this, I was wondering whether the success or the failure of a certain type of assessment stood on the quality of the video maybe as well as the thing being demonstrated. Is there a case for somebody being more effective in the actual video presentation as opposed to what they’re meant to be doing?

Graeme Knowles:

I think what I can say to this is I’m aware that using video has been very good for our diverse students who might struggle with having to do things directly in front of somebody might be very stressed by that process or doing it in a written form, which has been our historical approach to it. I think we certainly found it to be very useful in diversifying the assessment we give. The quality of videos is a bit of a double edged sword for me, because sometimes people and students are preparing a video for presentation, they can get a little bit over obsessed with the production values if you like, and so you need brief them very carefully and demonstrate how that isn’t the key issue here, is what you are communicating principally that’s the issue I think. Yes, it’s important to be able to, that the video’s clear and understandable, but I think we’ve got to be a little bit careful about overstressing that aspect.

Tom Henry:

I wonder if it’s a generational thing as we were discussing before this webinar that perhaps younger people may be more obsessed with the perfection of the video as opposed to the quality of what they’re doing. Josh, have you an opinion on that? Could that be a case?

Josh Beutler:

Oh I think we could make a pretty good assumption that there’s a little bit more obsession with perfection, just knowing the way things are represented. In fact, I’ve seen it and what we try to do in that process is allow the review of that video that’s being submitted or recorded for submission by that student in a very quick and easy way, but not have it too heavy and not have it too front and center, because we know Graeme, maybe to that same point, the repetition of recording or doing something and someone seeing themselves is that process before they even post it. There comes the reflection and that nice balance between saying, “Oh I really really need to do that again. I know I can do better.”

There’s independence and learning happening in a very authentic way there if a student were to record a presentation two or three times. That third or fourth presentation’s going to be much better than the first one they say, they close their eyes and click submit or something like that, but the obsession with perfection is something that we also, Graeme I congratulate you on that to make sure you’re clearly communicating that expectation to the learner to say, “Don’t worry about too perfect of a presentation. Show and let’s go from there.” And there’s a balance there that comes down to communication and design and making it easy instead of making it really challenging for someone just to do the very basic. We do see it though, that’s for sure.

Tom Henry:

I think they’re really interesting points and I think perhaps we could broaden this out to ask in general, what kind of impact does video assessment to video assessment systems have on students? And possibly on lecturers as well? Is it an overwhelmingly positive experience or are there challenges involved too? Sarah, perhaps I could ask you first.

Sarah Knight:

I think with any technology it’s being very clear at the outset both to students and to staff what the benefits of using that piece of technology is, and I think we see the thought too often that the technology is placed as the solution rather than actually understanding the rationale behind its use. I think for students and for staff it’s being very clear that they are able to get support, they know how the system works, they know the benefits of using it and more importantly are designing it effectively and pedagogically into their assessment practice. I think if those conditions are satisfied and the training, the support, the understanding is there links well into their assessment and feedback practice, then the benefits are going to be realized, and I’m sure Graeme will be able to give us some further insights into how that has worked at Warwick.

Tom Henry:

Yes, please do, Graeme. It’d be quite interesting to hear about it from not just the student point of view but also the lecture point of view and about operability and user friendliness.

Graeme Knowles:

I have to say this isn’t just an advert for GoReact, but we’ve found GoReact A to be very, very simple tool to use, but also the support that offer’s been very good. If we do get ourselves into a tangle, which does occasionally happen that someone could help us out, but in terms of staff, obviously I think we discussed previously that there’s been quite a challenging time for staff for the last two or three years. They’ve been through lots and lots of change, much of your unwanted and certainly most of your unprepared for, and so they are, that we can’t go to them and say, “Here’s another thing you should be doing.” Unless we’ve got some very good reasons for that.

I’m looking in at WMG, I’ve got a team of around eight people who are education designers or technologists who can support staff, but we always start with the why. Why are we doing this? Why is this worthwhile? What’s the benefit both the students and also to member of staff? And then we can also say to them, we have a team of people here who can help you to develop your thinking in this area rather than you being just given a piece of technology and asked to sort it out. That for me, I think the point that Sarah made before is really important that this is about design. If we design how we are students going to learn and design how assessment plays a role in that, then everyone can see the benefit. If you don’t do that, then it becomes an imposition for either staff or students or both.

Tom Henry:

Yes, exactly. Jenny and Josh, just over to you. Really about the GoReact product, I guess you’re fine tuning this all the time. How important is the feedback from students and staff and other users in terms of what you bring to the table going forward?

Jenny Gordon:

Fundamentally the experience that our partners and customers have with our product is paramount important to us and to feedback, but it has to make sense to them so that it works as part of their teaching and learning process. We spend a lot of time working with instructional designers and teams of people who have much more experience than we do at the subject matter and the delivery, and we make sure that our product can integrate with the fundamental foundational platforms that their learning management system and things like that, so that teaching staff and students don’t have to log into somewhere different, they don’t have to remember something else, so it becomes part of their existing method of working. The feedback that we get around how easy that is or how we can scale that up for our larger institution, that sort of thing, it’s really important.

And then in terms of feedback around the functionality, our starting place is that the user experience has always got to be very simple, because we need the barrier to use any product for an educational benefit, we need the barrier to use that very, very low, so the user experience is fundamental. I think the most important thing is that ongoing relationship that we have with our partners and our colleagues in different institutions in Graeme’s team and our team in the UK speak very regularly to make sure that we can support in the way we can, and I know that that’s the way that we work, but that ongoing feedback, that ongoing relationship is fundamental to both us and our partners.

Tom Henry:

Josh, I was going to ask you, and I think Jenny just touched on it there. Is operability ability and user friendliness, however you would like to term it, is that at the forefront of everything you do with GoReact?

Josh Beutler:

Yeah, if you want things to go poorly, introduced a little bit of friction into the process for the instructor or even the learner, and that’s the philosophical aspect we have to own as a technology is do not step in front of the learner, don’t make the instructor require or need training. That means when it comes to the way we work in LMS even it is instant. We have installations of GoReact where we support over 30,000 learners and instructors on an annual basis, and you cannot make it hard and it has to be designed well as well.

And even in the product, we’re in a healthy obsession where we’re obsessed with hearing what people think in the moment, so any user can actually submit a request or any sort of, ”Hey this right here, this is something I don’t understand.” And we catalog and categorize thousands and thousands of requests of what our learners and our users in general need as we look at our roadmap, and we ongoingly look, we’re always watching what’s being used and what isn’t, and we may remove functionality from time to time. Even if a small fraction of people are using it in the name of safeguarding the user experience and not have it be cluttered or have people locked up in functionality limbo of, “I’m not quite sure how this works.” And so that’s where our product and the obsession with feedback in our own world has to come to play, so it’s a great question, Tom.

Tom Henry:

Okay, thank you Josh. I’ll just make another quick broadcast to our audience. Please if you want to ask the panel anything about any of the topics we’ve discussed so far, we’d absolutely welcome that, and the chat box is there for you to drop any questions and comments into it that we can deal with as we go along. Let’s move along to the suitability of video assessment. Is it applicable to pretty much all higher education university courses or does it really benefit some over others? Jenny, perhaps I can bring you into this first of all.

Jenny Gordon:

Well obviously I’m going to say that it’s applicable to everything, because I think, but actually where we commonly work, we work with disciplines that have a vocational aspect to them quite often. We mentioned teacher training, we’ve mentioned nursing, education, communication, business studies and presentations, that sort of thing. Where there is a demonstrable requirement from that course we see a very high uptake, but the more that we are accepting video as a medium, if you like, institutions, faculties are saying to us, “There are other non-formal things that we would like to students to self-reflect on. Some of the soft skills, the ability to have a debate or a discussion, the ability to potentially talk to a patient, for example in some sort of role play exercise.”

So there are lots of different areas that we are seeing uptake of the use of video in that more formative, less structured environment where we’re encouraging students development of soft skills as well. That’s every everywhere from of post 16 further education and those apprenticeship places that Graeme mentioned into degree subject as well, but also in any professional development areas, anywhere that we want our existing in service staff to reflect, share best practice and develop, we see an uptake there as well.

Tom Henry:

Okay. Sarah, what do you think? Is this something that you could employ on a classics degree just as much as an engineering degree, do you think?

Sarah Knight:

I think it’s all about encouraging students to engage more with feedback and that process and looking at ways too, as Jenny said, scaffold that self reflection, and I think that any tool that can enable students and staff to think more about their practice, to engage in that deeper learning is certainly going to be beneficial. I think it’s being… I think in one sense that we often find at Jisc is case studies and examples of how others are using tools does help encourage other discipline areas to think about the applicability, and I think it’s the same I think with what we are talking about here. The more examples and the more benefits that we can draw out from both students and staff across different discipline areas, the more we can encourage, take up in wider areas. It’s all about the best practice, it’s all about encouraging what works and the benefits of it and how we can then transfer those benefits across to other subjects in other areas.

Tom Henry:

And Graeme, from your experience, how would you answer this?

Graeme Knowles:

Yes, I think it is applicable to all types of degrees, and if you think about a classics degree, you might think about I’d know Socratic inquiry and things like that where we’re questioning assumptions and so on. Actually the videos in that can, or video feedback you can use in this platform can actually help that. As you’re looking at a presentation, you can actually just say, “Did you really mean that? Is what you think about that?” And you can offer those challenges, and I think that there is potentially an application everywhere. We’re finding it particularly useful for our degree apprenticeships, and also we’re looking at micro-credentialing going forward and it has a real potential I think for the micro-credentialing end of things, but as I said before, I really see as potential for helping develop reflection in our students, and that’s got to be across every discipline that we can thinking of.

Tom Henry:

That’s an interesting point about micro-credentials. How do you see that working in conjunction with video?

Graeme Knowles:

Well it’s just really about demonstrating the skills and understanding that micro-credentials are looking for. There’s one particularly I think neat way of doing that, and again, this thing about having that personalized feedback, all that stuff I think plays well into that micro-credentialing agenda really.

Tom Henry:

Okay. May I ask the panel, do they think there’s a danger or perhaps a challenge that increasing remoteness is diminishing traditional face to face experience or feedback for students? And is that really an issue? Josh, could I bring you in on that one?

Josh Beutler:

I think there’s always the strive for balance between face to face interaction or remote instruction or teaching and learning, and I think the better the technology becomes at connecting those two in the middle in a balanced approach is always going to be the preferred method. Yet just the question of access alone to learning and assessment as learning, and to be able to have access when maybe you don’t have the opportunity to have a face to face experience. It just requires and it begs the experience to be as good as possible

And there will be a day sooner than later when we even know about micro-credentials, and that proof that Graeme talks about as far as being able to demonstrate is part of the credential and it’s part of the portability of that credential to say, “This credential is not just something that I have that is in a digital way, but here’s what it looks like, here’s what I do and here’s the proof that I have of being able to have those competencies. Look for yourself and see that I have that micro-credential, not just based on the credential itself but the actual evidence.”

And so I think that everything needs to be balanced, but I think the technology can shoulder a lot more burden to make things simple, to make it much more focused on skills and practice, to reduce workload of both the student as learner and the instructor, the lecturer as well. We feel like we’re just beginning in this process, which is why we’re excited about the future and creating more traction and more support around this topic.

Tom Henry:

Thank you, Josh. Sarah, what do you think? Do you think there’s a risk of deep personalization in a rush to implement and incorporate such technology?

Sarah Knight:

No, I think all our research and we survey students every year. This year we had over 32,000 students respond to our survey on their digital experiences and expectations of the use of technology, and we had roughly a half and half split between those students who wanted to be back purely on campus and those students who wanted that blend of digital and face to face and a small percentage of students who wanted to continue learning solely online, and I think we need to recognize that students’ preferences are changing. Many students have found the flexibility that digital offers to be of great benefit. It cuts down on travel costs, it allows them to perhaps fit their learning into work or caring responsibilities. We know that the cost of living crisis is going to cause even further disruption for many students who will find it increasingly difficult to travel into campus every day.

We do need to be offering that blend and we need to be offering it in a pedagogically sound way, and looking at where technology can really enhance and add to that value as we are seeing today in the discussion that we’re having. No, when used well technology can offer a very personalized and equally high quality experience for all students. Also take into account though, and one thing we haven’t mentioned though is obviously international students and their expectations and experiences of technology may differ perhaps to our students that are based and studying in the UK, so we also need to be mindful of the different backgrounds, the different contexts and different preferences that students are coming in with, but certainly not a second class experience at all.

Tom Henry:

That’s an interesting point. Could you expand a little on that? Do you see big differences in approaches and attitudes and uses of technology between countries, between cultures?

Sarah Knight:

Well I’d be interested in others views on the panel members, but we are just starting a piece of research into exactly that, to have a greater understanding of the experiences of our international students studying in the UK. Many international students come to the UK with an expectation of what traditional higher education will look like, which may be a very didactic experience from some of the countries that they are coming in from, and hence expect that they expect to still be in large lecture theater with that transmission of content. It is important for us to recognize the different preferences and cultural backgrounds that our students are coming in with, and again, to be very clear about the rationale from even looking at prospectuses for international students and being very clear around what that offer is going to be for students wanting to study at the particular university, but I’m sure others on the panel will be able to bring in their experiences on this topic too.

Tom Henry:

Sure. Well I was going to ask Graeme, is that something you’ve experienced, Graeme? Have you found differences between users?

Graeme Knowles:

Yeah, I don’t have anything scientific here, but it’s not anecdotally we do see some differences. There are certain nationalities who regard online education as very, very inferior to face to face education, and breaking that assumption is difficult and it probably wasn’t helped by the panics panic turn to online education that happened during COVID, that there is a little bit of that. I think there is also, as you say, yes, the access to devices and experience of learning is very different across different nationalities, and so what they bring to all either face to face or online learning is very different and you have to be flexible, but as you said, set the context and as Sarah said, set the context and clarify for them what they’re going to be experiencing.

Tom Henry:

Are you finding that in general most students are very happy to use this form of video feedback or are there still objections or grumbles in certain quarters?

Graeme Knowles:

So far we’ve not had anything very significant in terms of grumbles. Obviously the odd student might not be quite very happy with the way something went, but in terms of this is an approach, we certainly haven’t had that. I think that’s the reason that, well I hope the reason that’s happened is, because we’ve been very careful how we designed these interventions and how we design the students interaction with the technology and the students experience of the assessment. I think in terms of practical question that was asked initially, the remoteness thing I think is sometimes it’s a little bit of a red herring. I think it can be because historical education’s always been about communal learning and personal remote learning, individual learning, whether it be sat in a library by yourself or sat at home reading books, degrees traditionally, so it’s always been about that.

I think for me the key difference is the design, the thing is it’s about design. If we design it carefully, whatever technology we use, it’s part of that design and we’ve thought it through, it will be fine, but if we haven’t thought it through and interesting, one of the things I think that came out for us during the pandemic was that we hadn’t done as much design of our learning experiences and assessment experiences as we should have done. Lots of great lecturers who can paper over the cracks by being really charismatic and engaging in the flesh and so on, but they hadn’t necessarily thought through how they were delivering learning to their students, so it’s a great experience. Was it great learning? It’s a different question I think, and so what we’ve been forced to do by going online is to actually design things much more careful than we had before. And I think that’s been, we’re carrying that process through now into the blended learning going forward.

So we’re not going to go back to where we were before because we’ve learnt too much I think, and gone through a lot of pain with it, but I think that’s always been a little bit, education’s always been, there’s always remoteness. There has always a little bit of isolation, but it is a question, how do we design communal experiences into our learning process? For example, one of my colleagues had a video presentations to their students as a Netflix party. He got everyone to turn up virtually, get popcorn and see as a night the films rather than historically come to the classroom, watch should the people present, and so has tried to make it more of an experience in that way, so there’s lots of ways you can build community here.

Tom Henry:

And what a great point. The encouraging creativity in that way really helps to bring on board, like you say, the lecturer who thrives in front of an audience and perhaps feels a little at sea with the type of technology that we’re talking about. These systems obviously where that they’re been implemented, they’re being used, the take up is good. Where do we see ourselves in five years in terms of video based assessments? Josh, could I ask you that?

Josh Beutler:

My mistake, I think Graeme, your comment around design has me thinking as well, because the designing and iteration around design is just starting now when it comes to cycles and moving through and designing things, and I think the future of a video assessment even in the next few years comes down to can the assessment system itself and the learning experience excel itself inform continuous improvement in the design so that as things become designed over and over as things are continuously improved, they become better and better. And so for us it is not just that one moment, but at scale, if thousands of students are doing something similar or the same, can the data feed back into the system and inform the design process? And we’ve already had many, many programs look at the data holistically, longitudinally and say, “We’ve been designing this poorly for quite some time and we don’t even see what we expected in the baseline data.’

And the future is better design, better tools, ease of interoperability, and frankly the future of using video also includes the scaffolding that we’ve talked about as well. Where it’s not maybe just a presentation, but there’s preparatory work with that student to be able to say instead of writing a paper, you’ll write that paper, but then you’ll spend five minutes in presenting the key concepts in a clear format so that we can see, or you can show and demonstrate that it’s not just the writing that you’re actually you get well underway, but you can communicate those concepts clearly.

And then we see a lot of things in the future around the technology continues to evolve around the analysis of the video, of the words, of the analysis of the automated nature of it. Not to say that we should ever say that the system should analyze the video and audio and all the inputs and give some human-like feedback, but it should guide the process. It should inform the user, it should help streamline the workload of the instructor to say, “This video tool is not just providing the ease of use, but it’s also giving me a little bit of guidance into where this student might need some help and I’ll zero in there versus watching everything on my own.”

And so we see a lot of exciting things and a lot of parallels in education and not just in higher education but in further education, in trades in other places that it seemed very, very well prepared to start using technology to remove a ton of barriers that are just historically there and that they need to be able to be set aside.

Tom Henry:

Yeah, I was going to ask. In five to 10 years we may be seeing or we probably will be seeing an awful lot more video assessment within the workplace, if not already. Sarah, is this something that you’ll see will be a driver of technology in universities?

Sarah Knight:

Definitely. I think one of the things that comes through so often is the gap between graduates leaving and the expectations of employers and ensuring that our graduates have got the digital capability, the confidence, the resilience to be able to use the industry software that’s being used out in the workplace at the moment. That’s constantly changing, and we need to ensure that our students have a wide range of opportunities to use different types of technologies to be developing those employability skills of which to be able to present, to be able to self reflect, to be able to communicate confidently is all part of that set of skills that are required to be employable, so I think anything that can again encourage students to have those opportunities embedded within the curriculum is really valuable.

We had for many years that thing, “Well actually go off and do your employability or prepare for the workplace over there.” Whereas what we need to be doing is designing those opportunities into the curriculum along with developing their digital skills and confidence. Absolutely we really do need to be looking towards the future, and our jobs are going to continue to evolve and develop and students will have many different careers or jobs during their lifetime and we need to be giving them the resilience to be able to navigate through a changing global work.

Tom Henry:

Graeme, from your point of view, do you see that system of the more integrated systems becoming the norm?

Graeme Knowles:

Yeah, I think… It’s really, for me the evolution will be in how we use the systems more than the systems themselves. There will be how evolution, in the sense that as we try and use them differently, as Josh was saying, we’ll feed back and say, “Well I’d like to do this, but I can’t.” Making it easy for students to perhaps clip bits of videos and add comments and build that. That might be interesting to do, but I think really it’s about how we understand what we can do with these tools is the critical thing that’s going to take us forward. I think for me it always starts, the pedagogy always starts with the purpose and that’s where we need to, I think catch up with technology a little bit at the moment.

Tom Henry:

Okay. And Jenny, let me bring you into this finally, What’s your vision of the future for video assessment?

Jenny Gordon:

I think one of the things that Sarah said at the very beginning of the discussion, we were talking about enabling students or empowering students to have more control over their own learning as well. I think that a lot of what we’re talking about today is the instructional design and instructors and organizations incorporating video into their processes, but I’d like to also see students have the ability to record something themselves without being asked to do it, to start developing their own ability to self-reflect even before sharing it more broadly.

We talked earlier today about perfection and dumbing that down a little bit and normalizing things not being perfect, because part of the learning process is developing and getting better at things, so the more students can practice themselves in a way that they feel safe, that they can look at the potentially a rubric or a set of criteria that they should be aiming for and develop their ability over time, I think is really important. Bringing the ability to self-reflect using video that doesn’t necessarily fall part of a design process, I think is as important as having it in that more formal setting as well.

Tom Henry:

Okay, and do you think to that extent it could be and it needs to be supported by good training throughout?

Jenny Gordon:

I think that if… I think the simplest product, the simplest apps, and I watch my eight and four year olds quite ably able to record themselves or take a photo of something, give feedback to something, even if it’s in the form of a like button. I think that things have got to be easy going back to the lowest possible barrier to start using something, absolutely has got to be important. I think we’ve got to make it intuitive. We’ve got to always consider the safety of data and the recordings and things like that. That’s very, very important to us and meaningful ultimately, and I think underlying all of that is the data that comes out of something so that at the end of it we can use it to inform a learning process and help that student or another student along the way.

Tom Henry:

Sure. Sarah, is that something you could go with? Is that continuous training allied to continuous learning?

Sarah Knight:

Oh, absolutely. Definitely, and I really like to comment there, Jenny, about encouraging students to take control of their own learning. We do a lot of work with students as partners, students as co-designers, co-creators, and it’s really again, offering students to have that opportunity within a safe environment really. You’ve mentioned obviously the importance of data. We know there’s a huge amount of work to be done with students in ensuring that they are aware of copyright, aware of academic integrity, all of those compliance and those aspects that again, are critical when they’re move into the workplace. It’s really being able to give students the opportunity of that learning in that safe environment and then being prepared well for their future careers.

Tom Henry:

Okay, lovely. Thank you Sarah. Just looking at the clock, suddenly at the end of our hour, which seems unbelievable really and it’s always the sign of a good and interesting discussion that we’ve gone through it or seemingly so quickly. I’d just like to thank the panel very much for their participation and their views and their opinions today. As we know the webinar is or has been recorded, and that’ll be available on demand to look at, on demand on the EdQuarter site, but for now, thank you very much from me and thank you to Jenny and Graeme, Josh and Sarah for participating today. For you the audience, I hope you’ve enjoyed this and got something from it. Please don’t hesitate to drop us a line with any comments of views or feedback, shall we say, assessment of what you’ve seen today. I’m sure that we’ve all enjoyed this experience, so thank you very much and we’ll hope to see you again some time.

Sarah Knight:

Thank you. Bye.

Tom Henry:

Thank you. Thank you. Bye-bye.