Smarter Skill Assessments That Signal Readiness

Heather Lund:

Welcome to our webinar today. We are thrilled that you have joined us and hope you enjoy the presentation and walk away prepared to inspire your students and make a positive impact on their careers. My name is Heather Lund. I’m on the Gore React team and we’re hosting today’s presentation. For those of you who aren’t familiar with Gore React, we are a skill mastery and assessment tool used primarily across campuses in the US and uk. I’m happy to be joined today by our panelists, Sarah Baker and Gail Largent from Oklahoma City Community College. Before I hand it over to them, I’ll walk through a few points of housekeeping. Today’s event will last about 45 minutes, that includes 30 minutes of presentation and 10 to 15 minutes for q and a. We are recording today’s presentation, so if you need to hop off before we finish or if you’d like to share the recording with a colleague, we will email it to you.

We do want today’s presentation to be as interactive as possible, so throughout, please participate in the polls and prepare your questions. To submit questions, please use the q and a function. We’ll answer as many questions in today’s session, and you’ll also see a chat function. Please use this to introduce yourself, tell us about your school, which school you’re with. If you have links or relevant resources to share with other attendees, please do so in the chat. And if you experience any technical difficulties, please use the chat to reach out. Before we begin, let’s start with our first poll question.

Okay. The poll is what is your biggest challenge with assessing skills? Looks like it’s a little bit different skills outside the classroom. We will give everyone a minute to answer. Okay. It looks like a large number of you, the majority. It’s simulating real world scenarios with the second coming in as providing timely feedback. Thank you for your responses. Now, Sarah and Gail, please introduce yourselves and tell us a bit about your experience.

Sarah Baker:

Sure, Gail, I’ll go first. Okay. My name is Sarah Baker. I am a speech language pathologist and I am program director of the Speech Language Pathology assistant program here at OCC, and that’s what we call Oklahoma City Community College. To narrow that down a little bit, but we are an online program. We with a clinical component, so for five semesters our students do their didactic academic work in an asynchronous online community and then we send them out to do clinical experiences in the location that they live in. So even though we’re online, they still find their clinical spots near their residence.

Gail Largent:

Alright, and I’m Gail Largent. I’m a professor for the occupational therapy assistant program here at Oklahoma City Community College. I’ve been teaching here two years. I do have 26 years experience of being a certified occupational therapy assistant and I teach all of the comprehensive skills for adults and I started, the biggest thing is right now we’re all on campus starting in January of 26, we will have one cohort or track. Right now it’s called the Flex and it’s for evening classes, and we’re going to have that one be hybrid and they’ll come onto our campus once every four weeks for a skills checkoff. And so right now we do all of that on campus and they go through three different semesters where they have a comprehensive checkoff at the end of every semester and they do all of this right before they go out into their field work.

Heather Lund:

Thank you. I’m curious, when you’re designing assessments that evaluate skills, what do you prioritize and does that change whether the student is online or in person? Now both of you, Sarah, yours are online, but Gail, yours are all in person, but that will change. So how do you see that changing as well?

Gail Largent:

Right now, of course I have a rubric. Whenever I have any kind of Gore React assignment, whatever, I have a rubric that they have to follow and when they’re practicing their skills, that’s when they’re using the Gore react. And so I have them film themselves doing out whatever it is, if we’re practicing transfers or dressing following the rubric that’s uploaded on Gore React. And then after they do their skills checkoff or that lab checkoff is what we call it, then they look at their video and they kind of assess themselves. And then I go back in and look at it and then I do an assessment

Sarah Baker:

And our assessments. So in an ideal world, we would have them here on campus and have almost our own clinic. A lot of our grad programs have that, but we don’t have that luxury before we send them out to a clinical site. So we have to make sure prior to that that they do have at least the minimum set of standards before we send them off campus to a real life situation. And so we use the same standards that we would use if they were here on campus. I’m a strong believer and then it shouldn’t be any different. My student who goes out should not be unrecognizable or recognized by the fact that they were online versus on campus. So we’re strict, we stick to those standards and those competencies. And Gore React has helped us do that in a much easier way and to give better feedback because we could have them video and set up these case studies and scenarios that we’ve set up for them, they’re able to, like Gail says, practice those, get the feedback over a series of assignments. And if there’s places that we feel like they’re not competent in, we’re able to go back and remediate. We’re prior to sending them out into an internship offsite.

Heather Lund:

Thank you. Our next poll question, which tool or method do you use most for feedback? So seeing what the most common piece of feedback is for our audience today. I’ll give you just

Speaker 2:

A second.

Heather Lund:

Okay. Well, it’s pretty close right now. The majority 50% are written comments on assignments coming in second at 38% are in-person meetings. So a lot of just that written comment and in-person meeting. Thank you. Now let’s see. Sarah, what are some of the biggest challenges you face when assessing skills asynchronously and how have you addressed those?

Sarah Baker:

So of course some of the biggest challenges for me is that because we are doing this video wise, the way that we have this set up is that over the course of a semester they have these three mock therapy sessions that they have to do and each assignment doubles endpoints from the last one. So we don’t expect them to be great on the first one, but we do expect them to improve on the second one. And then the last is their final. What I have found is the biggest challenge is actually the students taking this seriously, that first video, we have a set of standards, we make sure that they know that this should be just a therapy session that they would be doing. And I’ll still get students who are wearing a tank top and don’t have their badge and they’re just kind being a little laissez-faire about it. And then they get dinged really hard with that and then they learn. So that’s a big challenge for me is that they don’t initially feel like this is as serious as it should be. So that’s probably not what everybody was expecting me to say, but that’s the truth. They feel like this is just a play assignment and those things don’t matter when. That’s a very professional lack of skill when you’re not showing that. So that’s my biggest challenge.

Heather Lund:

Thank you. That is interesting. Gail, what about you? What are some of the biggest challenges you face when assessing skills in person and how have you addressed those

Gail Largent:

Using Gore? React has really helped because what we do, we have for most of my classes, which I teach five different classes, most of them, I have four labs throughout the 16 week class. And so that’s when they use the Gore React and majority of the classes is why they’re doing their lab because I have 20 students and so it’s hard for me to be in the lab and be able to look at each student as they’re doing their skills. And so I have found this has been the biggest help, but getting them to, like Sarah said, for them to take it serious instead of trying to hurry up and get it done because it’s an assignment. But what I’ve found, especially this semester, again, I started it last semester and the biggest change I’ve noticed this semester is that they are already critiquing themselves.

Before I even get to go in there and make a comment, they’re already sending me a comment, oh, I know my body mechanics wasn’t good. Oh, I know I shouldn’t have let go of that patient. So they’re going back because these students that I have in the final class before they do their field work, I started them with the Go React class last set last year. And so they know how I’m going to critique them. And so they’re already critiquing themselves. And so hopefully we can continue this when we go to the hybrid classes with our one track that we have. But that’s been a wonderful tool. It really has. But again, at the very first, when they first start using it, them taking it serious.

Sarah Baker:

And to piggyback off that, Gail, and we do find that I make them do a self-evaluation before I will even give them their evaluation. So I don’t want them to read my feedback. I want them to do their feedback first and they are their harshest critics. And then they start recognizing small errors, things where they can improve and things like that. Next year I’m implementing a peer review and peer feedback into this same thing. So they will get paired up and have to evaluate themselves, evaluate a fellow student, and then they get evaluated by me.

Gail Largent:

Yeah, that’s something I hope to include too. Sarah and I was discussing this the other day, and so I want to include that too, especially in the last semester before they go out and do their field work, that’s when their skills have to be really, really fine tuned. And so I would love to, I’m going to try to implement that next semester if I possibly can.

Heather Lund:

So it would be just one of those three sessions for you, Sarah would be a peer, you’re just combining it or adding another assignment or activity? That would be a peer.

Sarah Baker:

It would be the same three. We probably do it over across all three.

I’m still working out in my mind the details and how I’m going to get that to play out, but the research behind peer assisted learning is there and I think it would just be another valuable tool for my students. And both of our students are in careers where they have to be supervised. So that’s another skill not just to be competent in what we’re doing, but also to work as a team and be supervised and be able to do that. So to give feedback to a supervisor and to receive feedback in a way that is constructive.

Gail Largent:

Yes, and I think the learning, the biggest thing is them being able to take that feedback and the constructive feedback because sometimes they’re just really hurt and especially they feel like I’m going to grade them if I’m giving them that feedback, it’s going to be a part of their grade or whatever. Whereas I think if it peer to peer, I see that being such a better learning opportunity for them.

Heather Lund:

Thank you. I like that. It’s always nice to expand on their skillset in various ways, even if they don’t recognize that it’s letting them grow as well through that peer review.

Sarah Baker:

Yes. Wonderful. And that’s not a skill that’s easily assessed how well we take feedback.

Speaker 5:

Yes.

Sarah Baker:

How well we interact with a supervisor.

Heather Lund:

Correct. Yes. Great. Our next full question feedback most what kind of feedback do you find most effective for learners?

Speaker 2:

We’re getting some answers

Heather Lund:

In. Oh, okay. So 50% real-time feedback during skill practice, which isn’t always available or an option. So let’s just throw that question back to our panelists. Sarah, what kind of feedback do you use in your programs and what do you find most effective?

Sarah Baker:

Well, I use all of those with the exception of real time, but with the Gore React and everything’s timestamped. So as I make my comments in Go React, the student knows exactly where and they’re watching it at the same time so they can see, oh, okay, right here, this child was struggling and I didn’t recognize that and I didn’t level up or level down. And I’m able to tell them that What I like about it is that I have some of those markers. I use your markers with some very specific things that I’m looking for and they’re looking for, but I will also use that audio or the video option quite a bit, especially if it’s something long that I want to explain to them or show them or they could have queued the child differently or something like that. So I use all of them except for that real time. Real time. They get the real time feedback in their clinical experiences outside of in their internships when they’re paired with an SLP in a facility somewhere or a school system.

Heather Lund:

Thank you. And Gail, what about you?

Gail Largent:

I use all of it Again, of course, the real time being right there hands-on would be wonderful if you could do that to every single student, but there’s just not enough hands in the classroom, not when I have 20 students and they’re all practicing at different points and everything. I can’t go and give them realtime feedback. So that is really, really the big thing with Go React that I love is I can stop that video when they’re getting ready to do something that I see is unsafe and make a comment on what they should do. And usually what I’ll tell ’em, okay, what did you think about the situation? Do you feel like this is safe? What could you have done? And then they will send me back, what would they have done? I mean, yes, they realize they were being unsafe, this is what they should have done instead. And then I do after the video, after they do their comments in the video and I do my comments on the video, we do look at the video again and sit down and we talk about it and stuff. But I use every single one of ’em. I do the audio, the grit, and I use it all.

Heather Lund:

And how do you ensure that your learners have the skills they need for success after they graduate?

Gail Largent:

After they graduate? Usually most, I hate to say this, but I’ve been in the field. Oklahoma City is a very small field when it comes to occupational therapy assistance and everything. And so since I’ve been in the field for so long, I know most of the people they go to work for, and I can tell you, and I think I have to contribute some of this to go react, is that in the last two years they have done nothing but praise how our students are coming out of this college that their skills, how their skills are, and the big improvement they said over the last two years.

Sarah Baker:

Wonderful. Yeah. So we’re using Gore react mostly to make sure that those skills are ready or they’re ready for those skills prior to them putting in a clinical experience. They do two 16 week clinical experiences at two different sites, and they’re in the real world. They are with an SLP at a school system or a private clinic, both of those 16 weeks. So our evaluation on that end looks like their supervisor, their clinical educator at that site letting us know that they have met the set of skills, they have an evaluation to use. Where we use Go React in those situations is because yes, we are online and so we have students all over the state and sometimes out of state. And how do you do a clinical site observation when everybody’s so spread out? So we will use Gore React to do live observations with our clinical coordinator on one end and set it up with the clinical educator and the student, and we can watch the student perform a live session live virtually. And because we’re doing it live and through Gore React, then we know that it’s HIPAA and FERPA and COPA compliant. And we’ve explained that in detail to all of our clinical educators who agree prior to that before going out, but we don’t record those sessions. It is just alive.

And

Heather Lund:

Are you finding that students are more prepared as they’re going into that clinical experience, Sarah?

Sarah Baker:

Absolutely. Like Gail said lately, and I think it’s a multitude of things, but Goer Act has helped us implement some better policies with that. But yes, our students are, what we’re hearing from employers are that our students are coming out career ready that they didn’t need a lot of onboarding. They didn’t need a lot of extra supervision in the beginning, that they were ready to jump in and start that job.

Gail Largent:

And I just wanted to say our program too, they do two eight weeks clinicals before they go out into the workforce. And so again, the same thing. Everything’s in person here. We don’t use Go React for any of our onsite visits or anything like that, but they do go out and do clinicals before they go out into the workforce. And even their fieldwork educators, they have seen a big difference on how they’re coming in and being more prepared.

Heather Lund:

Thank you. We have our next poll question for the audience. How confident are you in your current approach to preparing learners for real world success? Okay. Oh, an overwhelming amount, 67% somewhat confident with a tie between very confident and non-confident. So thank you. Now Sarah and Gail, you’ve talked about the approaches and the clinical experiences, but are there other approaches that you’ve found are most effective in simulating real world scenarios for your students before they get to that clinical or site practice? And can you give us some specific examples?

Sarah Baker:

Well, you want to go Gail? No, go ahead.

Okay. This is one of the places where it is just hard to simulate real world speech and language is not a, disorders in communication are not these obvious things and it’s very brain-based. And B, is it hard to come up with a mock child to a children don’t make their own great actors and actresses. So you have to use adults a lot of times who are given a script to be like a child. And that’s just hard. So simulations in speech and language are super difficult, I think. And we’ve used videos of real sessions and had students go back and critique those, but you just run into lots of issues with that. So that is a huge challenge I would have to say, is getting those and the ways we’ve had to be creative, overcoming that we do have some software that has simulations on it and we’ll start students on that. And then what they do with Gore React is that they get case studies and they have to almost write out a script to do that. So they’re not getting that real world until they actually get to their clinical experiences. And so we just try to make sure that we’re doing everything we can to get them ready for real world in a way that they’re still being guided along in those clinical experiences. Was that all over the place on that one?

Heather Lund:

I think that’s very helpful. Thank you. What about

Speaker 2:

You

Heather Lund:

Gail?

Gail Largent:

We have the same problem. I mean, it’s very, very hard to simulate someone to simulate what a real CVA patient, how they might react in a situation of getting dressed and stuff like that. So a lot of times what we use, we will use former students or we’ll not use students from the first semester to do simulations with the third semester. And so it’s all about them trying to act, but none of ’em can simulate. It really is in the real world. One of the things that we’re kind of fortunate to have is we have in our nursing program we have mannequins and some of the mannequins, you can have them talk and everything. So we get the opportunity sometimes to use the mannequins to simulate a little bit of real world. But there’s this really, to me, there is no way to really simulate the real world, but we try to set up all the different scenarios as much as we possibly can, what they might come across when we’re doing our comprehensive checkoffs.

Sarah Baker:

So I wanted to touch upon how we use this to build a more critical thinker

Because you’re never going to save the same child twice. The same child can come in, and I keep saying child because mostly what I do, but they can come in on a different day and look like a completely different thing. So you’re never going to be prepared for all situations coming through. So using this feedback and having students go back and to me is building those critical thinking skills which are going to be key to be successful out in the real world, going to be the same, nothing’s going to be textbook. No. And you’ve got to be prepared to adjust to the situation.

Gail Largent:

One of the new things that I’m doing this semester with the Flash right before they go out into their field work, they’re out their field work in spring is they have to do a 30 minute treatment session and they will be recording it on and then we will go back together, go look at it, we’ll make their comments, and then I’ll look at it and make my comments before, before they ever get their final grade. So that’s something new I’m going to try, I think it’s week 15, so that would be in December is when we start our checkoffs, comprehensive checkoffs.

Heather Lund:

Thank you. We have our next poll. How do you currently assess skills in your program? So kind of relates back to one of the previous

Speaker 2:

As well.

Heather Lund:

All right. The overwhelming majority, 67% of in-person evaluations, no one responded with video-based assessments or simulations or mock scenarios. But we did have a tie as well for combination of methods and other, so very interesting. Okay. Sarah, how do you make sure that the feedback you’re providing is meaningful and timely, especially in a remote learning environment?

Sarah Baker:

Well, so when we do these go react, I try to get them graded within usually a week because these are done in an eight week class and there’s three of ’em due. So you’ve got to get ’em that feedback in there before they start their next video. And that’s a mock therapy session is what that is. So you’ll react helps me do that by the way. And to get it in there, they have to do it, they have to evaluate, they have to give their own feedback, then they get my feedback, my rubric is all in there as well. So it is just a matter of that. We’ve had an increased number of students lately. And so what I’m looking at fingers crossed, is that I get actually a co-teacher with this class because as Gail said, if you have 20 students, it doesn’t sound like a lot, but when you are really wanting to give great feedback and personalized feedback, it can be a little overwhelming. So to give the quality feedback that I want to give, we’re probably looking at splitting some of this group up and having two different instructors give feedback.

Gail Largent:

Yeah, that’s probably the biggest challenge because I teach three different cohorts. So I teach all of ’em are at different levels in the program. And so altogether it’s like 45 students across the three different cohorts. And then on each of those cohorts, it could be anywhere from 10 videos a semester I’m looking at per class. And so yes, it does get very time consuming. Try to, like Sarah said, I try to get ’em all graded and everything within a week. This semester I’ve gotten so much better about spreading it out instead of the first time I did it, I kind of had it all bunched together and so it was taking an hour, one hour to do the feedback. And so now I have it spread out. So I have a little bit every week. So I have one every week throughout the whole 16 week course, which makes it so much manageable. I can manage that, I can get it done. Usually it’s within three or four days after the lab or whatever. I can get it the feedback to them, but it’s a learning curve.

Heather Lund:

Thank you. And Sarah, you had mentioned the hipaa, FERPA and Kapa and where both of your students are practicing and demonstrating skills. I know Gail, you’re not using Gore react in that clinical setting, but when those videos are reviewed by others and skills are completed outside the classroom using that video, how do you maintain compliance with standards security like hipaa, FERPA, and Kapa?

Sarah Baker:

So anything that we’re doing with an actual client, we are not recording. So we’re only using Gore React with a live feed. So we’ll start it, but we won’t record so we don’t have to worry about it that way. So everything else we’re doing, if I’m uploading an actual video of let’s say me doing a therapy session, then I’m making sure that the parent of that child has signed off on that and we keep all information protected health information off of it.

Heather Lund:

Okay, thank you. We do have one more poll question. What would most improve your skill assessment process? Okay, the top response was easier ways, let’s see, easier ways to simulate real world scenarios, 67%. And I think that you both had mentioned it’s a little challenging, hundred percent agree with that. Thinking of every single scenario or item or thing that could come up. So just preparing your students for that skill and critical thinking like you mentioned. What advice would you give other instructors across any discipline for assessing skills and making sure students are job ready?

Sarah Baker:

Good advice, Gil Louis. I think not underestimating the power of feedback. I know we’ve said it probably over and over and over again, but it is crucial. Students don’t know where they’ve made a mistake until someone tells them and in a way that is conducive to good learning. And I tell my students that because that’s what they’re doing in the career, in their career, they have to give feedback to the student, to the client on where their error was, how can we improve this? So I think it just kind of goes hand in hand that we have to give good feedback, have this to get good skills. And so there’s, my thing is don’t underestimate the power of feedback and good quality feedback.

Gail Largent:

I totally agree with that. And one thing that I have found is that if I’m giving, I’m right there on watching them do a skill and I give them feedback, they’re automatically, they tense up and everything and I don’t think they really hear it as much, but when I do the same exact wording feedback on Gore react, they’re like, oh yeah, I saw that. And so I think that’s really crucial. The feedback is so crucial for them to really hear it. And I always try to make sure that I let them know it’s not right or wrong. Everything they’re doing is going to be either safe or unsafe. And so that kind of has helped them a little bit. Whereas they’re even using that terminology now, oh, I saw what I did, I didn’t do it safe, this is what could have happened. But again, when I do the Gore React videos, they feel more freer to go and explain everything. And when we’re in real time right there in person, I think they feel a little bit intimidated or something and they don’t want to really express themselves as much as they do. And so again, one thing that I’ve noticed in the group that I started the first class I started this in, now that I’m in two semesters later with them, they are feeling more comfortable talking to me in person and taking that feedback from him in person. Whereas the first semester they were not.

But I think the feedback is very, very crucial.

Heather Lund:

So that feedback through the video and Gore react has kind of helped open that line of communication in person as well,

Sarah Baker:

Opens up that line of communication.

Speaker 5:

So

Sarah Baker:

When they do go to our fieldwork experiences, and we’re hearing that also from our clinical educators is that our students are asking questions more, wanting to get that feedback from and receiving it more easily

Gail Largent:

And wanting to be better clinicians, wanting to be better is what I’ve seen.

Heather Lund:

Thank you so much. I am going to see if we currently don’t have any questions, now’s the time. If you do have any questions for Sarah or Gail, please ask them in the q and a section while we’re waiting for some questions to come in. Is there, I know Sarah, you had mentioned you really use and the markers with in Gore React. Is that one of your favorite features or tools?

Sarah Baker:

I don’t know if it’s my favorite, but I dunno if I have anything’s favorite. I do like it though. So we have a textbook that uses these 12 therapeutic specific skills and is things like pacing and things that we could see throughout a session. And so I’ve just made a marker for every one of those. And then the students can go through also and just click on it if they see themselves doing it. And what I’ve seen is that if there’s some gaps and they’re like, oh, well I didn’t do that, I didn’t set up expectations at the beginning, but they could just real easily go back and do that. It’s good basic feedback, not very specific, but still all these things need to be in my session and it’s easy just to click on it rather than typing it all out

Heather Lund:

And a good reminder of the steps or processes that, the flow or structure. Right. Okay. And what about you, Gail? Is there a favorite feature or tool within Gore React that you or your students?

Gail Largent:

I have to admit, I haven’t used any of the markers yet, but the reason I continue to use it and I, I use it not just for skills checkoff, I forgot to say that the very first semester they have to go on gory and make a video of themselves explaining what occupational therapy is. And so that’s kind of where it starts. But it’s being able to stop the video when something is not safe. Because even if you’re in real time, and maybe I saw it out the corner of my eyes, but I wasn’t able to stop it right then. And so I think that’s the biggest thing that I love about using React

Sarah Baker:

Is they can go, we use it in multiple ways. So we’ve done it for assignments and we have a phonetics class where they have to transcribe a child with disordered speech, so they’re able to see the video and then they have to video themselves actually doing the transcription. So instead of proctoring something like that, which I don’t like to proctor, I don’t like to do online proctoring. And the students are still kind of a little time crunched to do it and they’re getting videoed, but they could always go back and it’s kind of funny, you could see their faces, they’re like, they’re listening again and then go back. So it’s pretty, we use it that way or they have to look at a video or critique another video and then we do it, or they have to make their own videos and come back. And then we’ve used it in a live observation session.

Heather Lund:

All complimentary for that learning experience and skill and critical thinking as well.

Speaker 5:

Yeah.

Heather Lund:

Well, thank you Sarah and Gail for this excellent presentation. You have provided a lot of valuable information and shared a lot of key takeaways, whether traditional online or hybrid, you can design assessments that accurately evaluate learner competencies and skills, providing more chances to practice, reflect and evaluate progress, and ensuring that learners have the skills they need for success after they graduate. Thank you to all of our attendees for joining us and making it an interactive presentation. Hope to see you on a future Gore React webinar and have a great rest of your week. Thank you so much.