Crafting Clinical Skills: The Art & Science of Recording RN Skills for Assessment

Presenter:

Dr. Dara Murray is an Associate Professor of Nursing at The University of West Alabama. She primarily teaches in both an on-campus associate degree program and an online RN-BSN program. Her research interests lie in the areas of nursing skill development, integration of technology into nursing education, and preparing new nurses for transition into the workforce.  Dr. Murray has eight years of experience in nursing education, and over 20 years of nursing experience. She received her Ph.D. in Nursing Education and Administration from William Carey University in 2021.

Transcript:

Dara Murray:

… Objectives today is just that each of you at the end of the presentation will understand the advantages and challenges of GoReact, gain some insight into the rubric development process, recording procedures, policy implementation, all of those things that are necessary for effective online evaluation nursing skills. And that you’ll acquire some practical knowledge through examples that I am providing you from our own grading rubrics that we use, our student instructions and then the frameworks of those policies that we have in place in order to guide our students through the process. Just a little bit of background, our university is located in Livingston, Alabama. It’s a small town located in a rural area of Alabama. It’s the diverse black belt region of Alabama. If you’ve never heard of it, you can look it up. There’s a lot of historical significance here and throughout the black belt. As I said, it is very rural, so some of you that are on the presentation from more remote locations or deal with some issues that we face that are unique to rural populations, I hope that you will benefit from this even more.

There’s some perks to using an online tool. 50% of our student population at UWA is considered minority. That’s identified as a minority student population and Sumpter County is where we are located and it is currently ranked the third poorest county in the state of Alabama. I tell you that just so that you can see that picture of where we are and some of the challenges that we face. We are not teaching in a perfect world over here in West Alabama. We have challenges just like all of you have. The challenges may differ slightly, but we do have obstacles to overcome and are constantly trying to come up with new ways to help our student overcome those obstacles. I had a poll for you guys and you can still do it I guess even without me being live, but I just wanted to know how many of you currently evaluate your skills in person and how many of you evaluate your skills using video?

Just so I could get an idea of how widespread the use of video is. I know video has been around in skills evaluations and nursing education for a long time. I know way back when I was working on my master’s, I had to record a video of a physical assessment and upload it to a YouTube channel. I had to create a YouTube channel, upload the video. That was quite a learning experience for me, but I could see the benefit of it. It allowed us to do it remotely, but you didn’t have all the perks of LMS integration and rubrics and all of those things. So just in curiosity, not necessarily using GoReact, but do you use video or do you evaluate skills in person?

So before GoReact, we were evaluating skills probably much like probably a lot of other nursing education institutions across the country. Our instructors first demonstrated those skills in the lab, in-person hands-on demonstration, and then the students were allowed practice time. So they were allowed time in the lab to practice those skills on their own or with an instructor if they felt like they needed a little extra help. And then we provided schedules, sign-up sheets, various ways that we did this over the years, but we scheduled students to come into the lab. We had a printed out rubric that we used a checklist, so to speak. And as the students performed those skills, we evaluated them in person in our lab rooms. Our students did not receive feedback on those skills evaluations until they had completed the skill. Meaning that if they committed an error midway through the skill, we had to just allow them to finish the skill as it was. And then after they were completely done, we would go back and highlight the areas that they had made mistakes or committed those errors.

So why did we change? Why did we decide that we didn’t want to do it in person anymore? Why did we implement GoReact? Well, one of the issues that we were faced with here was limited lab space. We do not have an overabundance of lab spaces in our building. And so as we were admitting more students, we found it difficult to schedule practice times and evaluations to accommodate all of our students. So just that’s a physical resource issue that we were facing here and we were trying to brainstorm ways to fix it, ways to get around it, so to speak. Another thing that we were realizing is that we had some grading inconsistencies. The faculty are grading solo alone, where I would be in a room with a student by myself. And I had to make a decision before I left that room as to whether or not the student passed or failed the skill.

And just based on feedback from students and even conversations amongst faculty, we realize that we may have some inconsistencies and things that I may have been failing students for, other faculty may not have been. Even though we had a rubric that should have ensured that objectivity and consistency, it wasn’t across the board. I don’t think that it was terrible, but we did realize that it could be better. Ineffective feedback, and this one is huge for me. I think it is so important. There were situations where I had students who committed errors during skills checkoffs and I couldn’t stop them at that moment. And a lot of you that have evaluated skills in person know what I’m talking about. You’re watching and they make a mistake and you want to correct them so bad because that’s what we do as teachers. We constantly provide feedback and let’s fix this and hold your hand this way, not this way.

And we couldn’t do that. We just had to wait until they were finished because you want them to finish the rest of the skills so you can see. I think about just intramuscular injections. We’ll just use that as an example. If they have incorrect placement of the injection and I stop to correct them and I stop the skill, then I don’t see if they recap their needle or not. I don’t see if they position their patient back correctly. And so waiting until the end made it hard because from time to time it was inevitable. You had a student that said, “I didn’t do that. That’s not where I placed my hand. I didn’t do it that way. I did it correctly.” Or even it’s hard to go back and just kind of remember exactly all the errors, especially if there were multiple errors.

Sometimes it was difficult to really give that good quality feedback. Another issue that we face was time constraints. We had three, just for example, I think we had three faculty that were responsible for grading five different critical skills and 60 students in one semester. That is a lot. That is a tremendous time that we spent physically standing in the lab waiting to evaluating students and waiting for them to arrive. Skills were scheduled at different intervals, depending on what they were. And so if we had a 30 minute time block allotted and a student finished early, then we were standing around waiting for the next student to show up at their appointed time. Or if a student was late or if a student didn’t show up, that was time that we spent just kind of wasted waiting on students to arrive at their scheduled time.

So just some challenges that we faced when we decided that we were going to use GoReact, these are just some of the things that we found that could have been an obstacle. One is the cost. It’s not free. GoReact is, I think it’s affordable, but it is not free. And so there is an added cost, so our students had to take that on as part of their student fees. Technology was another question that we had. It requires students to have a smart device and a reliable internet connection. Living in a rural area, an internet connection is sometimes spotty at best. So that was a challenge we knew that we would face. We knew that we would have to revise our processes and instruction related to these evaluations. We would have to educate our students on how this is going to happen and train them, so to speak. And then our policy, we needed to make some changes so that we could ensure that our students were graded fairly and that they were provided an opportunity to record their skills successfully.

So here’s the process that we used as far as integrating GoReact into our program. We began with policy. We took our current policy, we read through that policy and then just made very minor edits in order to reflect the recording process. We then took our skills checklist or our grading rubrics for all of our clinical skills. We made revisions to those. We had existing skills checklist. We compared those to some ATI skills checklist, looked at evidence-based practice. We used it as an opportunity to go back and make sure that we were using the most current evidence-based practice. And then we looked at them from the standpoint of will this work if the student is recording as opposed to a faculty member being present. Then we knew we would have to orient our students. We could not just throw this new product at them and expect them to just adapt immediately, so we developed a student orientation. GoReact was helpful with that, but we actually developed a slideshow.

We have a PowerPoint presentation that we go through with the students. We post that in Blackboard so that they have our policy. They understand our checklist, our processes and then the actual use of GoReact and how to use the tool. Then we provide instruction. In-person teaching with these skills evaluations, we did not eliminate that from our processes. We had that when we were doing it in person and we have maintained that. We still provide one-on-one teaching for these skills and demonstrations. And then we record. Once the students have been oriented to the process, they have reviewed the rubrics, they have received instructions, the students record.

They submit, we grade and they receive feedback on those skills. Now, what do we like about it? Here are the advantages that our faculty have determined that our probably our top three. So if we had to choose three things that we say are clear advantages of making the swap from in-person evaluations to video evaluations, this would be it. It is flexible. We love that word. We love flexibility. We love flexibility in the workplace. We love being able to have a little wiggle room, so to speak.

So students and faculty experience greater flexibility. The students sign up, we have sign-up sheets on lab doors and online Doodle polls and things like that. They get to choose their time. So they look at their class schedules, their study schedule, their work schedule and they can schedule skills performances in our labs. Some of the skills performances they can actually do at home at their own convenience and they don’t have to wait on us. So our schedules don’t have to line up. My students aren’t limited to my schedule. They can accommodate their own schedule without having to match it up to my free time, to my lab hours. Accessibility, they can access GoReact 24 hours a day. And so can I, as an instructor who is grading. It’s there. And so if at any point they want to go back and re-watch one of their videos, they can. They can upload at 11:00 PM or 11:00 AM. It does not matter.

And then I can grade the same. I can grade sitting at my desk. I have graded in an airport. I have graded in the car. Not while driving obviously, but I have graded these videos from wherever I am from whenever I have time. So it’s accessibility is one of the top things that we appreciate about using this. And then this last one. I think maybe as far as teaching and learning and truly crafting clinical skills, I think this is the piece that makes the biggest impact, is enhanced feedback. I talked a little bit about a challenge in providing feedback during in-person skills evaluations. The timestamp feedback that we have with the GoReact allows us to be able to make those corrections and highlight those errors at the exact moment that the error occurred. When the students see that, they can see. Okay, now I see it.

Now I understand what she means when she says it because I see my hands. I see where they are. I see exactly what I did. Right here is the error. And I’ve pulled students in my office before and pulled up their videos and said, “See, let’s look at this together.” Right here is where I’ve said that you broke sterility. Do you see why? Let’s fix it. Now you know. Let’s fix it. Let’s make it better. I think that crafts the skills throughout this entire process. I think that’s probably the biggest positive. I included a couple of pictures. I did take off the name and I’ve tried to get slides or little still shots that did not have students’ faces, but do you see here on the left-hand side of your screen? That student had not injected air into the bowel before she drew up her medication.

And so you can see where the faculty that was evaluating this skills video said draw back one milliliter of air and inject to the ketorolac vial before drawing up. So she had just gone straight into drawing it up. And so that’s an example of one of those timestamp feedbacks. On the other side of the screen is a physical assessment video. You can tell that that looks like it’s probably in a dorm room or someone’s bedroom at home. We allow them to record that skill wherever they would like, but see, they’re checking pedal pulses here using their thumbs. And so the evaluating instructor made a comment right here. Use your fingertips, not thumbs. And so I just wanted you to see if you’re not familiar with GoReact sort of what that timestamp feedback looked like. The longer skills videos, especially physical assessment, is so wonderful to be able to add this timestamp feedback so that they can go back and see all of our comments and improve that skill.

I told you guys I would talk about policy a little bit. Here is two snippets. I mean, this is straight from our actual policy. So if you want to read over that, but it’s just a small excerpt from our critical skills policy. That’s what we call it. I don’t know. Different people have different names, but we just added in that statement in red that says including video return demonstrations. We did not completely rewrite this policy simply because we implemented a video recording tool. All of our key components for our critical skills policy still stand. It did not fundamentally change that policy. We just simply added in a statement saying that video return demonstrations just to let the students know that that is a possibility. And then the use of GoReact is more detailed on the syllabus. So we did not want to be super specific here because we didn’t want to back ourselves into a corner.

We still evaluate our vital signs check off in person just because we use a vital sign mannequin and it requires that we are in the room setting the vital signs on the mannequin. So we haven’t quite figured out how to do that one via video yet. And so because of that reason, we didn’t want to exclusively say that all skills would be recorded using video, so we kept it very simple here. Here’s an example from our syllabi. We only use this in the courses that we use GoReact in. Obviously not every course has skills evaluations, but the ones that do, this is the snippet that we have on the syllabus for that. And you see it’s pretty simple, too. We list GoReact as a required resource in all the courses in which we utilize the product, so it is listed under required resources.

We list it right there along with our ATI, our required textbooks, things like that. And that’s just a little snippet. I wanted you to see an example, but it is short. It’s to the point and it just tells you that we use it throughout the course to complete critical skills. And then it tells the students about setting up their account in GoReact. This is from our foundations syllabus and it actually lists the specific skills that we require GoReact to be used for. You’ll notice that vital signs is not there and that’s because we do that one in person. And then instruction, so how do we do this? How do we instruct our students? Well, first we assign ATI skills modules as tickets to the lab. I don’t know how many of you use ATI, but they have wonderful skills modules where they have … I’m giving a little plug to ATI here. If any ATI people are on here, they have great skills modules with videos and rubrics and evidence-based practice and they’re great.

So we assign those, so students have to complete those modules before they come to lab. That way they have some background knowledge before they get to us. Then we provide hands-on demonstrations with small groups. This semester, for example, I had two separate lab groups and each lab group had eight students in it. And so I would take my lab group of eight students. We would go into a skill server room and I would demonstrate a skill and then they would practice for the rest of the lab. And I was floating in and out, making sure that if they had questions or if I needed to correct techniques, that I was available to do that. And then the students have access from day one to the GoReact rubrics and to those ATI skills videos. So both of those things they can refer back to as many times as they would like.

And honestly, that faculty-led instruction, we believe it is still the key to success. We had one of our skills this semester. We had an ice storm and so the university suspended operations due to weather. And our faculty instruction was cut short and the students struggled more with that skill than any of the others this term because they didn’t have that. So that hand-on instruction we still believe is the key to their success in achieving success with the performance of skills. This slide, I just wanted to give you some examples. In the top left, we use Blackboard as our LMS here at UWA. And you’ll see that’s kind of how the course coordinator for our foundations course organizes those critical skills. The very first thing is the GoReact orientation slides. We go over that with them in class, but they have access to that PowerPoint presentation that they can pull back up and make sure that they understand the processes.

And then she has listed out a folder for each of the critical skills. And so there’s lots of information housed within Blackboard for the students, so they know from day one what is expected of them and they have all the resources they need to be successful. The bottom left where it says critical skills videos, basic guidelines, that’s a snippet from that orientation slideshow that we have on GoReact. And that’s something that we created just to sort of walk the student through step-by-step exactly what they’re going to have to do and what it should look like. And you see that last bullet point, view your video and critique it prior to submitting.

Watch it. You get to watch yourself, you get to record it, you get to watch yourself and then you get to choose whether or not you submit it. And then on the right-hand side, that’s just a very tip top of our skills lab schedule. Students are given a schedule on day one of the course overview that says this is what is expected of you. Skills lab one, before you come these are the module skills modules you have to complete. That is your ticket to lab. It has meeting times, it has dates. If there is a skills checkoff, then that is listed as well. The date and time, the deadline for the submission of those videos is on that, too.

Rubric development is really important. Just as advice to any of you who are preparing to begin skills videos, your rubrics need to be clear, objective. It needs to be something that the students can look at and know exactly what’s expected as far as grading. They should be able to look at this rubric. They should be able to watch their video and look at the rubric and know exactly what their score’s going to be. And so we worked very, very hard on our skills rubrics to be able to make sure that we captured everything on there that we needed to capture in order to do a good job evaluating skills and give them good feedback and evaluate them fairly.

We took the ATI skills modules checklist. We used those. We used what we had historically been using in the past and we adapted our skills checklist to reflect any new evidence-based practice and the use of video. And then we took those skills checklists and we built them into GoReact as rubrics. They are added to the assignment. The student can view the rubrics and as well as the faculty of course. Here’s a little picture. I took a snippet of the skills module’s 3.0 checklist for IM medications from ATI. And then on the right side of the screen, that is actually a snipped little picture that I took from our GoReact rubric for that same skill. So you can see how we sort of adapted in order to make it a very clear grading rubric. So that when I’m watching the video, it’s easy to grade.

I can follow this checklist as the student progresses through their video. Student prep. What do they do? How do they prepare? How do they get ready to record? We don’t want them to just start recording. They’ve got to practice first. We don’t want them to watch us do it one time, watch a video and then go record themselves. We want them to practice. Our skills lab coordinator provides sign-up sheets. We are very fortunate to have a skills lab coordinator and so she organizes that for us. She provides sign-up sheets. She makes sure that the adequate time is allocated in the labs for students to be able to go in and practice. The students are required to find a classmate to video their skills. We do not do that for them. And then it works. I know that we questioned in the beginning if they would do this, if they would find a classmate, if they would communicate with each other.

And by and large, we have. We have had very, very few issues with this over the years. The students can film practice videos. So while they’re in their practice time, they can record themselves. I encourage them to and then they can go back and grade themselves using the rubric. Our lab coordinators schedule testing times that are explicitly scheduled for testing. That’s when there’s a sign on the outside of the door that says quiet, recording in process. So we try to make sure that when they are recording, one, they’re not interrupted by another student and, two, that it’s quiet on the hallway so that they can … That increases their anxiety and interferes with their performance if it’s a distracting environment. So we try really hard to make those testing times explicit.

This is a hidden benefit. It’s that student collaboration, especially in the first semester. I watch them every year. They don’t know each other. Some of them don’t know anyone else in the room and this sort of pushes the envelope a little bit and makes them find a friend, partner up. They get to know each other. I hear them laugh about funny things that have happened while they’re in the lab practicing or recording each other’s videos. And so we think that is sort of a hidden benefit that maybe we don’t highlight enough, is that it forces them to get in those labs and rely on each other. They have to depend on each other to be there for each other. And like I said, by and large, it has worked really well for us.

Moving into evaluation, this is more of the faculty side of it. I grade videos as assigned to me by our course coordinator for the foundations course. We split them, sort of a divide and conquer, and I have my group of students that I evaluate on all of their skills videos. And it’s my responsibility to make sure their videos are submitted and then I grade them. Objectivity is maintained by exclusively using the rubrics. I told you we worked very hard on our rubrics and now we do not stray from those. If it’s not on the rubric, then it is not required. We worked very, very hard on those.

However, sometimes things are questionable still, even with our rubrics. And with the videos, one thing that I absolutely love is that we are permitted to collaborate when the performances are questionable. We have a brand new faculty member here at UWA this semester who has been … She’s working as a skills lab instructor. She’s helping grade videos. And I know that that is a big benefit, that we can call each other in and say, “Hey, would you look at this with me? I think this student broke sterility right here. Will you look at it with me?” Let’s zoom it in. Let’s slow it down. Let’s look at it together and make sure that we are grading the student correctly.

And then that last bullet point goes back to flexibility. I can grade at a time and place that is convenient to me. We are busy, we are stretched. We’re doing clinical, we’re teaching the classroom, we’re teaching in labs, we’re having to evaluate skills videos. We know how busy we are as nurse educators. And so with this, if I’ve got 30 minutes in between class, I can grade a couple of videos and then come back later and do a few more. And I wouldn’t trade that for anything. It has been a wonderful time saver in just to be able to be given that flexibility of being able to grade or if I’m off campus, I can still grade. It’s great. That’s been a very positive thing for us. So there’s a hidden benefit. Faculty save time. Fun fact, when I did my dissertation for my PhD a couple of years ago, I actually did my dissertation on GoReact.

And one of the most significant findings that I had in that research study was the amount of faculty time savings. In my study, we split a group of students into two cohorts and one we evaluated in person and one we evaluated using GoReact. And it was a significant time savings that we found. Faculty kept time sheets and we went through and we calculated the exact down to the minute how much time we spent doing each method. And it was a huge time savings. It was the most significant finding I had in that study.

And that’s a big deal to all of us. I mean, our goal is to craft these clinical skills and to help our students achieve success, but it is also phenomenal if we can save ourselves some time so that we can spend our time better doing other things. I can be mentoring students or working on lesson plans. It’s been a wonderful thing to be able to free up some of that time. And behind it, you see another snippet from one of our videos on the left and then that’s that grading rubric on the right. So if you’re not familiar with the platform, that’s what it looks like when I’m grading. I have the video on the left and then I have that rubric on the right so that I can go straight down that rubric, grading the student as they progress through that skills performance.

So I sent out a poll last week to our students who are currently enrolled in our foundations course and are using GoReact. They’ve just used it this semester, so it’s fresh on their mind. So I sent out, I did a little brief Qualtrics survey and sent it out and these are some of the comments that I got back from our students. So positive student testimonials. I like how easy it was to use and understand. We have a very wide range of students in our current cohort. We have traditional students that are 19, 20 years old and then we have non-traditional students that are in their 40s, in their 50s. We have students that already had college degrees and are back in school. We have got a very wide range of levels of being tech savvy. And so I liked that, that they feel like it’s easy. And then how you can comment at a specific time in the video, so that’s the timestamp feedback piece that we talked about earlier.

Being able to watch the videos before uploading. They get to perform the skill, watch themselves and then be confident when they submit that they’ve done it correctly and that’s a big deal to them. And then that last one that says it’s less stressful than having an instructor in the room. So this is the research study that one of us needs to do, is a qualitative study on student levels of anxiety. Get to get some feedback because we have heard that kind of over the course of the past few years, is that it decreases their anxiety. When we are in the room with them, they’re so nervous and you can tell their hands are shaking. And so they like being able to do that performance without the intensity of having an instructor in the room with you, getting only one shot at it. And so they liked that, too.

And then I asked them for a challenge, too? What’s your favorite thing about GoReact and what’s your least favorite thing? What was challenging to you? And these were some things that I got back from them as far as challenging, so they said trying to get a good angle so the instructor can see what you’re doing. That is a challenge. I agree with them on that because there are times. And this is if you’re not using it currently. Be prepared for this. You need to be able to see clearly in order to grade, especially sterility.

When we’re indwelling urinary catheters, we need to be able to see that entire sterile field. And if we can’t, then we can’t grade the video and they have to rerecord. When they’re inserting an IV catheter, we have got to be able to see the insertion site very clearly. If we can’t, then we can’t grade the video and they have to resubmit. So that is a challenge and we stress that during our skills demonstrations. And so we have to probably try to teach that on the front end to avoid these issues. Then they said getting the video to play clearly. Sometimes it is blurry, sometimes it’s their device, sometimes it’s their internet connection. That’s a challenge, then that someone said sometimes it messed up. They tell us. This is not all the time, but that’s with any other video.

It’s happened to me personally before. You get halfway through, you run out of storage on your phone and it messes up. And then the last one just says getting used to the program, but I think that’s probably the case with any type of new technology. You have that learning curve. So again, I’m so sorry that I’m not live with you today. It makes me super sad. If you have any questions about what we’re doing, this is my email address, my office number. I would love to connect with you. I would love to talk to you.