Online Skills Evaluations: Saving Faculty Time & Building Student Success

All right.

So we’ll go ahead. We’re going to go ahead and get started. It looks like we’ve got–

we got some people. We’ll have a little bit more filtering in, but we’ve got our people right here. Again, there’s that poll that will pop up, that yes or no question, please feel free to go ahead and answer that for us. But hello, and thank you for joining us at Reaction.

My name is David Peck. I’m on the success team here at GoReact. And I’m really looking forward to today’s presentation today.

Today with us as Dr. Dara Murray, who’s been with GoReact for a long time and so has the University of West Alabama, just really long time great users for us. And she’s going to be giving her presentation today.

If we want to go ahead, go ahead. And as we start, introduce yourself, share your thoughts or share resources in the chat window. Just make sure that again your message settings is to everyone, and so that you can share that with the group as we’re going through our presentation today.

If you have any questions for our presenter, we’ve got that Q&A window, and just go ahead and ask any questions there.

And then, if you see any question that you’d like answered, just click the thumbs up to upload it, so that we can filter through and see what everybody else would like answered. And we’ll just make sure that we answer as many as we can. We’ve got that 30 minute time window. So we’ll try and get to everything we can. All right, and with that, I’ll turn the time over to Dr. Dara Murray.

Hi, I’m so excited to be here today.

I am–

like David said, my name is Dara Murray. And I am currently an assistant professor at the University of West Alabama. And I have been teaching nursing for about six years now. And I have been a nurse for around 20.

So nursing education has just become my passion over the past few years and. In our partnership with GoReact has been so beneficial to us. And that’s what I want to share with you all today is just some of the reasons why the problems that we were facing before the implementation of GoReact, and then how the product has helped us fix those problems, and just some of the ways that we have found–

creative ways we have found to integrate it within our curriculum with our nursing students.

I’m going to share my screen.

I hope you all will bear with me.

And while I get this set up, it’s been a minute honestly, since I have hosted a Zoom. So our students have been back on campus for a while now. And so–

hold on.

Let me see.

OK.

I hope that is showing you my screen.

David, could you give me just a thumbs up if all is well. I can’t see my chat bar with these dual screens.

Good. OK, great. Then I will get started.

I have two monitors in here on my desk. So sometimes, my Zoom settings do not show just like I want them to.

So the title of my presentation today is just online skills evaluations. And I’ve focused that sort of on two separate areas. And that’s one is saving faculty time, which is very important to all of us and building student success, because that is ultimately the goal. And that is what we’re doing as nurse educators is trying to find new and creative and innovative ways to ensure that our students are successful.

I wanted to share with you about our University.

I’m an alumna of the University of West Alabama. And I have lived here in Livingston for the past almost 20 years.

So the community is near and dear to my heart. It is located in a very small town–

Livingston, Alabama. We’re about an hour and a half from Birmingham, Alabama and about an hour from Tuscaloosa which people have usually heard of.

So we are situated in the midst of Alabama’s very diverse Blackbelt region.

And from an historical perspective, the Blackbelt region has been very important in the Civil Rights movement. And just historically speaking, it’s rich and diverse and full of just a really interesting history.

It’s rural.

We live–

we are very spread out. It is like–

I said, it’s a small town.

Our surrounding counties are very rural, which poses some issues for us. We have patient–

I mean patients, excuse me, students who sometimes drive quite a distance to come to school here.

About 50% of our student population at UWA is minority. And that is also reflected pretty consistently in our nursing courses.

Sumter County, which is where we are located is ranked the third poorest County in the state of Alabama.

All of those things combined, we face some challenges. We have a large population of first generation college students. Many nontraditional students who are not living on campus.

They’re driving, commuting to campus. A lot of them have families at home.

And we see–

a lot of our students are from economically or even educationally advantaged backgrounds.

The objectives today I want us to talk about how to use video and timestamp feedback, and how that translates into a more successful and competent student.

When I look at using online video recording tools and how that enables faculty to become more efficient with their time. And I also wanted to discuss how video feedback impact student’s self-assessment with debriefing and also communication.

So we had some problems that we were facing here. As a nursing faculty, especially faculty who teach in what we consider our foundational level courses, where the majority of our psychomotor skills are taught, we were facing several problems. It was tedious. It was difficult. And those of you who are in the audience who are nurse educators, you already know that it is crucial to a nursing education program that you are providing adequate preparation and instruction and evaluation to ensure that your students are performing skills safely before they enter into the clinical setting and perform those skills on real people, on live patients.

There are certain skills here that we consider critical. I think it’s probably very common.

The ones that we use are probably similar to the ones that are used at other institutions. But we consider battle sound assessment, physical assessment, medication administration, indwelling urinary catheter insertion, and IV insertion as what we deem critical skills, meaning that students have to successfully perform those skills.

We call it a pass off for a validation in order for them to progress in the program. There are other skills that we teach, and how the students do return demonstrations on, but those are the ones that we deem critical.

But some of the problems that we faced here, the main problems is that we were spending an excessive amount of time in skills laboratories, just simply of not just teaching the skill, but just that evaluation portion of the skill.

The students expressed anxiety.

They were in a room with faculty, just one on one. And they had to perform. And most of you have probably encountered that with students or their little hands are shaking, and their faces turn red, and they’re extremely nervous.

And there’s a lot of research in the literature. If you know that I have looked and read several articles about ways to try to help that.

And instead, it just qualitative studies that show that the students express that it is very, very anxiety-inducing to go into these face to face situations with an instructor.

Another portion that we struggled with was that during these face to face evaluations, we didn’t want to interrupt the students. We didn’t want to stop them in the middle of their performance to provide feedback.

And so, we had to wait until they were completely finished with the skill, and then give them feedback on errors that maybe occurred during that performance which is not ideal. We have to write and make notes, and we have to go back.

And we have to try to explain it to the student.

And so it’s not ideal.

The other thing that we found that we were missing that we wanted to be able to provide was a way for students to evaluate themselves–

self-evaluation and self-reflection of their own performances in order to allow them to in a sense grade themselves.

Research tells us that, that self-reflection is a critical part of learning.

And we were missing that. There was no way we didn’t have cameras set up in our labs for them to go back and view their own performance and see it from that bird’s eye view.

This is our workflow previously. When we were conducting our skills check offs or validations, face to face in the skills lab, this is what it looked like for the faculty.

I feel like this is similar. I’ve talked to a lot of faculty at other nursing schools. And it sounds like a lot of us kind of do things the same way.

But the students would attend a lab, where we taught them how to perform the skill.

They have been provided videos, skills videos skills, checklist. So they receive all their information, and then they were scheduled for times to return to the lab, mandatory practices, where there’s an instructor available to watch and help and correct and make adjustments as needed.

There’s faculty in the room for both of those. And then they are scheduled for their return demonstrations or their skills evaluations.

And the way we did that was we just had sign-up sheet scheduled at 30 minute intervals. And the students had to report to that lab for that one on one demonstration.

And the faculty were in there for hours, because if I’m scheduled from 1 to 5 to do skills lab evaluations, I’m there. I’m not back and forth to my office.

Sometimes, students don’t come, and that you feel like that’s wasted time. But just to put it in perspective, at our institution, we have four instructors that were needed to perform all of the check offs.

We have about 60 students, typically, sometimes more, sometimes less in a single cohort.

We had five skills.

15 students roughly per instructor. And that’s about 37 hours of time spent strictly evaluating skills in one course in one semester.

That’s a lot of time.

And we felt like that time could be spent a little bit better.

Especially that downtime in between students come into the lab. Maybe someone finished in 20 minutes, but 10 minutes, is it enough to leave and go do something else? Or you wait? Or maybe there’s time to get theirs at 1 o’clock, but they’re late. And so you’re left waiting for 10, 15. Sometimes, they don’t show at all.

Following that initial evaluation, any students who were unsuccessful would have to repeat those skills, remediate and repeat. And so that was additional time as well.

So we started doing some research into solutions. How can we fix this problem? How can we make it better? Are there other ways to do this?

And so of course, we went to the literature. We went to the research. We looked online. We talked to other programs.

And so we started leaning toward the implementation of an online video recording tool. Some of the reasons I listed here that that’s the direction that we went in–

budget cuts.

We could not afford–

our administration tells us that we cannot afford to hire additional faculty. We can’t hire adjunct faculty or additional full time faculty to come in and to help us manage not just the teaching of the skills but the evaluation portion as well. So we had to be creative and figure out how to do this better without hiring additional faculty.

We thought about the geographical distance of that our students travel. So taking the faculty out of the mix and just thinking about the students and convenience for the students, we thought about that we have students who are commuting an hour, an hour and a half. Some of them, every day.

And is it worth their time to drive an hour and a half to campus to complete a 30-minute skills demonstration? When some of these can be recorded at home and submitted without them having to drive to campus.

Side note, I’ll add. This worked very well for us when we were all at home during the COVID pandemic.

We were so thankful to already have this in place during that time when we physically could not have our students on campus. Feedback, that was another thing that I listed as one of the problems that we were not able to provide really good feedback during the performance of the skill.

And online video recording tools when you have that video, it allows you to highlight and provide feedback throughout the video without that worry of interrupting the student.

Efficiency, which is what I have alluded to that tremendous amount of time that we were spending and some of that we deemed sort of wasted time, the waiting in between, and the time just spent standing and waiting idle time that was occurring with us standing in those labs for hours at a time.

And then we worried about skills preparation and the mastery of those skills. Were we are effectively evaluating those skills? Were we are consistently using the checklist appropriately?

Were we preparing our students? And then were we evaluating and were we truly evaluating that our students have mastered those skills?

And so those were some things that led us toward the online video recording tools.

We ended up landing on GoReact as the product that we chose to use. Our College of Education at UWA was already using GoReact.

And so through collaborative conversations across campus, they suggested that we do the demo. And we said if it worked for us, because it was working very well for them. And so I wanted to share with you some of the reasons. I don’t want this to be a sales pitch. But some of the reasons that we did ended up going in this direction, it was the online recording tool that we wanted.

So it was checking those boxes that we sort of had already formed in our mind that we needed and we wanted. And so it also–

the fees were minimal. And so we didn’t feel like it would be an extra burden on our students. And as I mentioned in the beginning, we have a student population that we have to be super careful about fees and the expense of our program. And so that was important to us as well.

The fees were minimal, and they were able to be paid through their course fees without the students having to pay additional money out of pocket.

The recording was easy. The videos were compressed, so they could use their cell phones.

They didn’t have to buy equipment. We didn’t have to buy equipment.

The videos were submitted to the instructor’s platform through an app.

It was pretty seamless. And then the students through the platform–

through GoReact, we were able to provide them instructions and a rubric. So they were able to see the instructions and the rubric before the skill, during the skill as they watch their own video, and then after evaluation as well.

And then the big thing that really kind of sold us on this was that we could evaluate these videos whenever and wherever we chose. So it is not uncommon for me to grab videos sitting on my sofa or in a hotel room.

I’ve done that.

I can do it quicker. And it’s a lot more convenient for me than standing for hours in a laboratory.

Some of the other things I wanted to talk about that are specific to GoReact or targeted practices, these are things that we have sort of–

that we utilize with GoReact, the students do like a sort of a ticket to the evaluation, where they have to evaluate–

self-evaluate a practice video.

So those aren’t a graded, but they are essential in providing a way for them to identify areas of improvement. Then the timestamp feedback portion, that is huge.

That is probably my most favorite thing is to be able to highlight an error at the moment that it occurs, and to be able to go back with that student, and say, let’s look at your video and Paul’s. You see right here, right where I comment, made this comment, that’s where you break sterility when you were inserting that indwelling catheter. Let’s look at this.

And so, it I can pause. I can stop. I can rewind and make sure that I place that feedback at the exact moment that it needs to be for the student to get the most out of it. Reflective commentary, so I put my feedback in as I’m grading the video.

And then the students are actually able to comment as well. And so they can go back, and they can watch their video, see my feedback, reflect on that, and add their own comments as well. So it provides some communication between the instructor and the students.

Rubrics, we always rubrics in education all the time for multiple different types of assignments. This is no different. The students are provided the rubric. It makes grading super, super simple for us to be able to just click through as we’re watching the video. In the beginning, GoReact helped us with that and put those, built those rubrics out for us.

Now that we’ve been doing it for a while, we just do it ourselves. It is pretty simple. And we don’t have any issues with that.

Markers are like stickers or kind of like little sticky notes. It’s the way I think of those. And I just use markers for things that I say a lot, like nice job here. Things that I find myself repeating over and over during those evaluation videos make markers. And so it’s just quick. And I don’t have to type out the entire sentence every time.

Documentation purposes, I love this feature as well. And I don’t like to talk about the negative side of nursing education and some of the things that maybe don’t go perfectly.

There are sometimes students who don’t agree with their evaluation.

And we have very, very clear documentation with these videos.

There have been very few times that I have had to do this, but it has happened where students are unhappy or discouraged or they do not agree with my interpretation of what they did. And so it provides a really great way for us to pull that video back up. We can have another faculty look at it, and we can pull that student in, and say, here’s your video. Your performance is documented. It is here. My comments are here, and it’s very objective.

And so it provides us a great way to go back and pull those records and look at that performance.

And it also will correlate to your LMS, so just that rubric, just the documentation of the grades. And we’ll also transfer.

But it is great to have that record of the performance in the evaluation in the comments right there in black and white.

Then uploading, uploading was simple for our students. They record through the web app. It compresses their video. It uploads to the site. We see it immediately, and then they also see our feedback immediately. So it was very seamless for us. And then of course, I don’t have it on here, but there was no expensive equipment.

And they go to the app on the website, it’s free. They didn’t have to buy anything additional.

They used multiple types of cell phones.

They have androids and iPhones. And they all seem to work OK.

I have a little video that I wanted to show you. For those of you who may not be using GoReact or any type of recording for skills video, it’s just a snippet. It is not the whole thing.

It’s just a few minutes of a skilled studio. This is IV insertion.

The student did sign a release, so she gave her permission for us to show this. And like I said, it’s just the beginning, and it’ll just show you how we do this, and what it looks like from the instructor’s point of view.

Hi, I’m Jessie Lister from the University of West Alabama, and I’m here to insert your IV catheter. Can you state your name and date of birth for me?

Janet Jackson, 2/3/71. Janet Jackson, 2/3/71. Great.

OK. Now, I’m going to get all of my supplies easily accessible. First, I’m going to attach my flush to my–

So that’s just a little quick snippet, but we have them go through the whole skill just as they would have done it if we were standing in the room for them. Another student videos them, and they’re given instructions as well. We haven’t had any issues with that. The students seem to work pretty well together.

And they know to move their camera, and they know to get angles that we need in order to be able to see.

This slide, this is just a picture that I did. I just snapped to my screen, because I could not–

the video, when I download the videos, they don’t show the feedback. But I wanted you to see what that looked like.

So for our skills videos, This is a physical assessment video and this student was using the wrong side of her hand to assess the temperature, and the skin temperature in the extremities. And she verbalized that the skin was warm and dry. But the instructor in this instance is saying OK, stop right here. Timestamp feedback.

Use the dorsal side of your hand. And so looking at that, the student can see in the exact moment that she used just a little incorrect technique there. And so it tells her right then in that moment how to correct that.

Other creative course assignments that we have done–

I talked about skills evaluations a lot, because that is the primary thing that we do. That’s what we spend the most of our time grading through GoReact is psychomotor skills. But we have implemented a few other more creative fun things that we’ve had our students do that they have we’ve got good feedback from the students and the course instructors. In our pharmacology course, they do pharmacology commercials.

And they are awesome.

This group that we have this year has submitted some phenomenal pharmacology commercials through GoReact. And I cannot tell you how creative some of them, where I was just blown away.

Safety hazard identification–

and so, I listen to the session before my session, and they talked about–

the presenters talked about a house of horrors that they created as a collaborative venture with a couple of other disciplines on their campus.

We do that as well, and we have used GoReact, except we don’t create a house of horrors. We ask them to evaluate their own home or their work environment or somewhere. We ask them to do just a quick scan of the room and identify some safety hazards that they have found. Therapeutic communication techniques, and I’ve got a video just a second of a teaching video.

But therapeutic communication teaching student interaction with not a real patient, but student interaction with either the interdisciplinary team or a pretend patient.

Another creative thing that we use during COVID is that we had them record a physical assessment on someone in their home, submit the video, and then we used our simulated EHR. We use EHR tutor through ATF.

And we had them document that assessment of that patient that they recorded into the EHR. And then we were able to not only just evaluate their performance, the video performance, but we were able to compare that to the documentation that they completed within that simulated chart. So that took it to another level.

We’ve done those things separately before. But that provided a way for us to see the assessment and see what they documented to make sure that it correlated appropriately.

Here’s just a very short snippet from one of those pharmacology videos. And I wanted to share this, because–

I’ve cut out a lot of the actual commercial.

But this was one of about warfarin. And there was a lot more to the video was longer than a commercial. Honestly, it’s like a little short movie clip. But the section that I pulled out, I really liked. Because it shows a student, and they’re both students, but it shows a student playing the part of a nurse, and another student playing the part of a patient. And they are doing patient teaching. And I loved this, and so I wanted to share it with you all.

Hey, Ms. Simpson.

Hey.

So the doctors put in the order for you to be discharged.

OK.

I’m just going to ask you a couple of questions about your medication so you have an understanding. OK. Do you know when you’re supposed to take it?

In the afternoon.

Yes, ma’am.

OK.

And do you have a Medicare bracelet or do we need to provide one for you?

It probably would be better if you all get it for me.

OK.

I appreciate that.

Also, you need to get a soft bristle toothbrush.

OK.

And also, you said, take it in the evening.

OK.

And not to drink it with grapefruit juice or any grape juice.

OK.

Or anything. And you know when to come back? When are you supposed to come back and check in?

The doctor said every–

come every week for lab work. But come see her in a month.

So that–

to me, that was so good. That was completely student initiated. I mean, the assignment was created, but there was not faculty in the room for that. They did that on their own.

And so it provided a way for them to do a deep dive into warfarin.

Those students probably could tell us anything we wanted to know about warfarin at this moment, and then I thought it was just so creative. And it was just a different type of assessment that our pharmacology instructor was able to incorporate into that course.

And that shows us too that they can use that–

we could create another assignment, where it is patient teaching, discharge instructions, pre-op instructions.

It kind of opened up a way for us. This assignment has led us into the creation of our future creation of more assignments in other ways to use the product.

So I just wanted to share that with you, because I thought they did a really nice job.

This is our altered workflow. This is what it looks like for us now when we evaluate skills now that we use GoReact. So the student’s videos, their own performance on their own terms outside of class time.

We have sign up sheets where they can sign up for a time to use our laboratory. If it’s a skill they can perform at home, they just do it on their own time. They have a due date.

The instructor watches the video and the rubric is in GoReact.

The students can self evaluate or not, and then we evaluate it and provide that feedback at the end. So it’s just a little four-step process.

It has simplified things for us so, so much. And so it just it takes out a lot of that time that we were spending just kind of standing around the lab waiting in between students.

I did a research study in 2021.

It was actually for my dissertation for my PhD. And this is just a little snippet of it. If you would like more concrete data from this study that we did here, I wrote about it in my dissertation, but there’s a lot more to it than this. I just pulled out this one snippet, because I thought it was huge.

This study, we only looked at the IV insertion skill, so this was not total skills. It was just one singular skill.

And there were only 37 students in that group. So our numbers were a little bit lower. But we split that group into two. And so half the group were evaluated just using that traditional method of face to face.

The other half were evaluated using GoReact.

And so the faculty spent 254 minutes evaluating the face to face group and only 176 minutes evaluating the GoReact group.

So it was proved to us. We knew it already, but that was concrete proof that our time was reduced.

It was the same group of faculty, and then like I said, it was the same course. But we split the students into a group A and a group B. And this was the results that we got.

I would be happy to share any of that data with you. Like I said, there is much, much more detail to that study. But I did not want to bore you with that today.

But I would be happy to share it, if any of you are interested.

Thank you so much.

I feel like I talked a lot. I feel like I told you a lot of things that we do here and shared a lot. I hope that I didn’t jump all over the place too much, and that you were able to follow most of that. I would be more than happy for you to ask questions now or the next slide is my email and my contact information. If you would like to contact me outside of this session, I would be more than happy to speak with you as well.

Thank you, Dr. Murray for that awesome presentation. We do have a couple of questions that in the Q&A box that I’ll go ahead and ask here. From Colleen Parsons, she asked, what do you think about using GoReact for SBAR nurse to nurse post-clinical reports to their clinical group.

I think it would be awesome.

I do. I think that is a great idea. I might write it down on my notepad. I think it would be great.

Like that last video where we were just showing patient teaching, I think SBAR would be great. I know personally, I take students into clinical.

And SBAR, they struggle with.

That hand off report, shift report or nurse to–

nurse to doctor, nurse to–

floor nurse to ED, ED to ICU. I think there’s a lot of different creative ways that you could implement that.

I saw actually with your workflow, you had students self assessing as well with I’m assuming you had student rubrics. Did you add student rubrics on–

Yeah.

We did. It’s an exact copy of our faculty rubric.

Oh, that’s awesome. So everything that they’re seeing from their instructor giving them feedback, they’re also able to give themselves that same feedback on that rubric. That’s cool.

Do you notice any difference with students that do fill out those self-assessment forms?

Yes.

They don’t always catch their own error. They don’t always see it, because I don’t think they always understand.

They don’t always understand sterility, or they don’t always understand. But they catch things, because what happens is they do that, and then they come to my office or they send me an email and say, I did this wrong or I went out of order here. Should I redo my video? Sometimes they know it’s crucial, and if they need to redo it. But I get a ton of questions from those students who are self evaluating because they catch those mistakes.

And they see it before it ever gets to us. And they recognize that oh, that wasn’t quite right. So yeah, it works it works well.

Awesome.

I’m a huge supporter of those self-assessment in GoReact. And I think that’s great that you use the faculty.

There’s one more question in the chat here.

Are your faculty members compensated for their time grading the videos? Currently, we use GoReact and in-person check offs. It still takes time to grade the videos.

No. We are not. I wish. But we are not.

It is just considered part of the course and part of the course hours.

So since we’re not compensated for those hours, it was–

that’s why it was when one of the reasons it was such a big deal to us to shave some of that time out of the skills lab, because a lot of it was just truly unproductive time that we were standing, waiting for students to arrive or waiting in between students. Well, when I’m grading videos, as soon as I finish grading one video, next, next next. And so I can go through them so much quicker, because there’s no time in between.

I can just go straight into the next. And another reason that I wonder in this kind of speculation, I guess, but I think we grade a little faster too, because there’s a feature in GoReact where you can speed up. And so in the South, we speak slowly.

I tried not to talk too slow today. But we do talk a little slower and our students do as well. And so I can 1.5 or 2 times the regular speed. And I can get them through the verbalization part of their videos a little bit faster.

I love that. I didn’t think you spoke too slow. I’m from the West, so we’re a little slow too. So–

[LAUGHS]

That’s awesome. Last–

I think, actually no, I think that was our last question there. Thank you so much Dr. Murray for your time.

And thank you for doing this presentation. Again, if you have any questions for Dr. Murray, she did have her information at that last slide.

So and she said that, feel free to reach out.

But thank you again for coming. Thanks for being a part of Reaction. And we hope that you enjoy the rest of the presentations throughout the week.

Thank you so much. Bye, bye.