Coaching With Purpose: Using AI to Unleash the Potential of Remote Asynchronous Video Observations

Dr. Allison Smith:

We are talking about coaching with a purpose using ai, and we’re going to be talking about our research and what has led us to using the AI in the clinical space with candidates. So we have Keith Walters here. He is the associate Dean at Cal Baptist University. He is the associate Dean of Educator Effectiveness, and he has been working in this space, doing this research for decades. Started in 1998 and that wasn’t a jab at your age, who

Were born probably. And I’m Allison Smith. I’m an associate professor at University of Massachusetts Global and I do research in clinical practice and video mentoring, video coaching, and the use of ai. And so we’re excited to share that with you, but enough about us. Why are you here? That will really help us with this presentation just to learn a little bit about who you all are. So we came up with four kind of big areas we thought you might fall into. If not, that’s fine too. So we’re just going to raise your hand if you feel this describes you. I am skeptical of using asynchronous video observation, and I’m here because my institution is making me move in this direction.

Okay, nobody, I love this next one. I am using asynchronous video observation, but I’m unsure if it is better than in-person observation. Okay, the next, I am excited about the potential of using AI during asynchronous video observation and am here to learn how to move forward. Okay, that’s the main group here. And the last is I am using the AI functions during asynchronous video observation, and I am ready to learn more. Okay. Sounds like we’re mostly number three in how to do this. Okay, that’s great. Okay, come on. Here we go. So we’re going to start with our research and then lead into our use of AI in video observation specifically with GoReact. So our research really has morphed and been an evolution. When we started this work on video observation, it was during covid and it was really focused on whether video observation was actually valuable.

Is it better than in person? Are we comparing the two? Can we expect the same outcomes with video? So that’s where we were then. So we did survey research on the perceptions of our university supervisors, our candidates through the internship route, through traditional student teaching, seeing how they felt about it. It really morphed then to more be how we’re using the video observation. And we were looking at current practices, looking at handbooks, how our clinical practice, how it’s organized and how it’s being delivered. And then it moved to really developing a protocol, which we’ve done through the research, and then how to use AI in tandem with the protocol to really improve coaching and really support our candidates to become great educators. And so that’s where we’re at right now. We are currently working on a survey, survey research IRB approved two institutions who that are using that protocol.

This summer we’re going to work on another IRB to do research on using ai, not just using it, but the impacts of using the ai. So that’s our next step. And then we’re going to do a case study on the university supervisor’s experience, because it’s more than the numbers, it’s more than the data. We want to know their experience. And so we’re moving forward with that. And now we’re going to talk about the different supervisory roles. So in our research, we really found that there are all of these different roles of the university supervisor. And so Keith came up with this great grid matrix that he’s going to talk through.

Dr. Keith Walters:

Okay. So I have been working on understanding teacher development since I was in my doctorate in the 1990s. And I have taken a very holistic synthesis approach to understanding the research behind teacher development, looking at things like emotional intelligence, positive psychology, core value reflections, professional learning communities. The list goes on and on and on. It’s very cross disciplinary in nature. And there’s two big things I want to highlight as we go into the research that seems simple, but when we take a step back are things that we oftentimes forget. Number one, most clinical supervision tends to be reactive. You tend to focus on rehashing something. Our protocol is built on being proactive. The second huge element within our protocol is this notion that if we’re going to create successful teachers, we need to help our new teachers understand how to positively navigate disappointment and frustration.

And those two little huge tidbits are the heart of what we’re doing. It’s a major paradigm shift that has a lot of power. So to understand how this paradigm shift works, we began to look at what are some of the tensions that educators face. One tension that supervisors face is, am I a friend of the person I’m supervising or am I a teacher? And sometimes we have supervisors who are reteaching our coursework during clinical practice because they think that’s their job. And then we also have supervisors with toxic, toxic positivity where you swear that their candidate is walking on water even though every kid in their class is out of their seat. So that’s a tension. The other tension that we looked at when we talk about supervisors is my role to be an assessor to determine whether or not you are doing everything correctly according to your state standards, or is my job to be an encourager.

And as we looked at this, we began to go, these tensions actually create a good grid matrix. Now the next thing that you begin to look at as you create the grid matrix is what are the different roles that a supervisor plays and the roles in the bottom left where we are, encourage your friend, that’s the caregiver. And then up on the top right, you have the teacher assessors, and that’s the gatekeeper. And what we’ve noticed is that most supervisors hang out in these two areas. And both of these areas when we talk to candidates, are actually ineffective in building candidate agency. And as we begin to look at this, what we begin to realize is the power comes when you are a coach or a project manager. And the promise is when you get down into that bottom portion where I am a co-learner with my candidate focus on advocating for equity.

So when we begin to look at the supervisor set of the supervisor role, we begin there’s, we realize there’s a huge untapped area that brings power to supervision. And so based on that, we created what’s called the tacking protocol. And I’m going to share everything I know about going in a sailboat right now. I know a sailboat does not go in a straight line, a sailboat tax back and forth. And that’s the reality of human development. And if you want to blow your mind, think about this for a little bit. In many instances, the worst lesson I would want to see my teacher candidate teach is the last one. And that’s really counterintuitive because usually we say that that should be the best one. But if you think about it, by the end of a clinical experience, the classroom is now more of the candidates.

The planning is more of the candidates, and the students are no longer in the honeymoon period with the candidate. So by the end of clinical practice, we’re beginning to see the reality of where the candidate should be. And the reality would be as a really first year teacher. So it’s not surprising in many instances if the last lesson they teach is in some instances the least effective lesson they teach. So realizing that development goes back and forth, we begin to say, what are the key elements? And element number one is trust, because we’re going to be talking about and normalizing disappointment and mistakes. And our whole protocol is about normalizing the fact that we don’t know everything. I don’t know everything. My candidate doesn’t know everything. The mentor teacher doesn’t know everything, so we have to normalize mistakes. Second, it’s all about adjustments, not change.

Adjustments, building off from the candidate’s strings to help them become a little bit better. Third, it’s about coaching. I have more experience and I’m going to use my experience to help you become better in the areas that you want. So I leverage the different experiences there. Number three, kindness. You’re really getting the candidates to be very vulnerable in understanding that they are not perfect. And so you’ve got to be very kind and aware of the fact that they are going to have emotional struggles. And the ultimate goal is ingenuity. If we can create advocacy and increase self-determination in our candidates, we know we have created somebody who can overcome a disappointment and actually learn to thrive. So that protocol is what we created to drive our video coaching sessions,

Dr. Allison Smith:

Which leads us to how we actually put all of this into motion. And this is done through GoReact. It’s the observation platform that we use. I’ve used different observation platforms where the video is uploaded and analyzed, but I have to say, I’ve been overseeing over a hundred university supervisors at once and the district mentors and the students using the platform. And I needed something that was easy to implement, something that I didn’t have to answer, a million questions and support, be tech support, be all of those things. And I wanted just a really helpful place where people could go if you have questions, and they will handle it because I do not have time to handle all of that. And that’s where GoReact really is where I find that solution. And the protocol with GoReact has been really useful. And the AI functions have been nice in augmenting the clinical practice experience for the candidates and for the university supervisors.

So when I’m thinking about AI use with GoReact, I’m thinking through the lens of program design accreditation. I need data, I need ease of use for all of the supervisors. I need ease of use for all of the students. And so these are the the big things I’m pulling out to just share with you in how the use of AI has really transformed how we coach our candidates automated feedback. So AI analyzes the candidates videos, provides instant feedback, so that’s really great. Their performance, it identifies strengths, areas of improvement. It also reduces the human bias. It’s a great third point and provides a great starting point for discussions. Sometimes candidates don’t know where to start, they don’t know where to analyze. And so it’s a really nice place to start with that data-driven insights. So we want a great experience, but we also, as programs need data programs can collect and analyze the data from the AI functions.

They can identify trends, patterns. So you can really analyze and then inform your program improvement. So you can pull that, those data talk with faculty on, Hey, you made this change in the coursework, here’s the outcome. Or I’m looking at these data, and there’s a lot of times when the AI is pointing out where pacing is off or some aspect of the teaching is off, how can we improve our programs to really address this issue that’s coming out in the field? We have data. It’s not just me saying it or someone or a group of people saying, guess what? I’m seeing this. We actually have data on this. The other part is accreditation. As someone who does reports for accreditation, I’m in meetings for accreditation, we use aquip, and then we have our state CTC accreditation. And so these data are really nice to show that you are actually looking at what’s happening in the field. The next is time saving. AI automates the feedback. So it frees up the supervisors, and I wouldn’t say just to give them free time, but then it allows time for deeper analysis so they’re not wasting their time on the next point, which is the common feedback responses. So if you’ve supported candidates, there’s just standard things that are happening often. And so let the AI handle that and let the supervisor dig deeper and look at something more critically. Yes.

So the question was, as the instructor, do you see the AI feedback? It’s a great segue to the next slide, which covers that. Okay, the nuts and bolts of the AI implementation, which I think was of interest. Now, this is very quick overview. We only have 20 minutes with you. We will be over at the GoReact table to share more about our research, our protocol, the ai, how we use it. We don’t work at GoReact, but we are happy to share how it works in the field. So the nuts and bolts, if you haven’t really used it, I just wanted to go through this. Candidate records themselves teaching. They upload it to GoReact, GoReact AI function, then creates a transcript of the video, then it analyzes and it automatically provides the feedback. But here’s where to answer your question. The supervisor can approve, edit, add to decline the AI feedback.

They could say, this is junk. Or if, let’s say it is junk, right? Have the candidate write why this AI feedback does not apply in the situation. That’s the critical thinking. Maybe AI doesn’t know the personal context of the community within the school, outside of the school. Having to justify why that AI feedback doesn’t apply is really getting them to think deeper. But as a supervisor, you could certainly say, Nope, nope, nope. I have seen people use the AI function and do zero 20 minutes of teaching 90 comments. Imagine you’re the teacher candidate. It’s overwhelming. AI function, in my opinion, is not something to just use on its own. You’re not just pushing a button and saying, go for it. You’re keeping it humanized, which, oh, I’ve got one more point here to share, but we’ll share some humanizing stories about AI if that’s a concern of yours.

The next part is these AI markers that gory act uses. And so here are a few of them. Video chapters, pacing, engagement, listening and feedback. They have more. These are the things you as the program or the supervisor can choose. You can say, I only want them to have AI feedback on engagement because that’s what we’re focusing on. So you can choose. I think it’s a really nice thing. And then on the GoReact site, you can also recommend other markers. And I think that based on those recommendations, they’ve come out with new ones. I just learned about new ones on differentiation. So feedback from the field. So if you have markers that apply to your institution or your state and you’re like, you know what? My state focuses on asset-based language. I’m going to go to GoReact to the website, recommend it, talk to them, and then they can put it into action. They’re open to doing that. So I really appreciate that part of it. And we are short on time, but we are going to talk about a humanizing AI use really quickly.

Dr. Keith Walters:

So one of the things I would emphasize is that the supervisor should carefully edit the amount of AI feedback that’s given, and I would actually encourage supervisors to provide no more than five to the candidate. And the reason for that is because in our proactive coaching session, we want to dig deep into what’s happening with the candidate and provide candidate structures to move forward with confidence. So I had a student last semester, and we’ll just call him Donny t, and Donny was one of these people that never made a mistake. If he made a mistake, he became paralyzed. Unfortunately, his fear of making a mistake made him incredibly nervous. So the first 15 minutes of every lesson, he was a basket case. The last 15 minutes of every lesson, he was actually pretty good, and he was just, I’m nervous, I’m nervous, I’m nervous. And as I was beginning to watch a video, certain things began to emerge with his commentary. So there was this particular video that he turned in and he goes, I am so sorry Dr. Walters. I totally missed the fact that the student right in front of my camera was watching a soccer game.

So we started watching that video clip and we started dissecting that video clip. And most of you who are experienced teachers will get this right away. The student was really smart who was watching the soccer game. He had one eye on the candidate and one eye on the soccer game. And as soon as the candidate came anywhere that was close, he switched to what he was supposed to be doing. It took me four minutes of watching and rewatching that video before the candidate realized, oh, this student is sneaky. And then we could begin to talk about what kind of strategies can we use to make sure that students are engaged? And one of the power of having, for example, five markers is after we dissected that segment, then we could go look at another one, and then I could have two more or three more markers that the candidate could reflect on when they’re home. And then after we realized this problem, we were able to have this very rich conversation about actually looking at student work and not at just what students were doing. That was a life-changing moment for that candidate. The next video, and talking to his mentor teacher, his nervousness went way down because he was now beginning to realize that it’s not all him. Kids are people. Kids are sneaky. And it’s not his fault if they’re smarter than he is. He just has to learn how to manage them.

Dr. Allison Smith:

And then real quickly, one last little story about just that was from the candidate perspective. This is from the university supervisor perspective, and I had this supervisor that was just very appreciative of a new program, a new protocol, a new way of doing things with GoReact and ai. And I just took a clip from one of the text messages he sent because I think directly hearing his words is the most useful. So he says, I look back on when I first started as a supervisor a few years ago, I felt that there had to be a better way to help candidates, but couldn’t put my finger on it. Then you came in and created a program that responded to all my concerns and even went above and beyond. I feel a much closer connection to my candidates and to me, we’ve become a team.

In fact, I was surprised to hear that my candidates message each other frequently discussing best practices, Cal TPA and other things. They’ve become friends in the profession. Amazing for that. Thank you. That’s from a university supervisor who has never met his candidates in person. They have only been supported through video use, AI meetings on Zoom, all of those things. So if you’re worried that the video observations, the AI feedback is going to dehumanize the experience, I hope that that gave you some hope or light that it doesn’t have to be that way. So with that, that is our presentation. Here’s our contact information. We’ll also be over at the GoReact just to talk, and I see friendly faces here that we know very well and other new faces. We’re happy to talk further about any aspect of this presentation. Thank you.