Rethinking Assessment in the Age of AI

Pete Morgan:

Hello and welcome to everybody on our webinar today. We’re thrilled to have everyone that’s joined us and hope that all of you will enjoy this presentation and walk away with some ideas and tools to help update your assessment strategies to drive growth through meaningful feedback and enable students to demonstrate their learning authentically. That’s going to be a real key point of what we’re going to talk about today, authentic assessment and AI strategy. My name is Pete Morgan. I am the strategic account manager for GoReact and handle everything at scale for the use of GoReact to be perfectly designed, to be impactful and helpful, especially in all these strategies. I’ll be hosting today’s presentation. And for those that are not familiar with GoReact, we are a learning technology solution focused on improving learning outcomes through the use of video in feedback and assessment processes.

We’re in more than 800 schools, colleges, universities, and professional organizations found across the globe. GoReact is incredibly impactful and we do a great work here. Now, before we begin, I’m just going to run through a couple of points of housekeeping for everyone. Today’s event will last about an hour. That leaves about 45 minutes for our panel discussion and our two incredible guests that are here with us and 10 to 15 minutes of QA towards the end. Now for everyone, we really encourage the discussion, the questions, because we would love to gain, not just the inside of what our panel discussion will carry, but also more specific questions from all of you in how you can improve your strategies here as well. That’s what our panelists are here for. We will be sharing out today’s recording, so be sure that if there’s anything particular you would like, everything will be sent out to you via the same modems that you were able to sign up here, and you can share those recordings out with a colleague after the email has been sent out.

We do want today’s presentation to be interactive, like I said, so throughout the presentation, please don’t hesitate to add questions to the Q and A, to pose questions through the chat. All of those will be monitored on our side and make sure that everyone is heard. We’re really excited to hear what you’re thinking and bring all of that into play. Submitting questions to the panel can be used through the Q and A function. Please include the panelist’s name if your question is directed to someone specifically. That way we can keep intact and make sure everything is directed to the right person, and we will answer as many questions as possible through the chat session even before we get to the Q and A, covering those aspects. Using the chat itself through Zoom, please use that to introduce yourself. Now I’m seeing all sorts of different entries from people as we’re coming on, which is great to see.

Tell us where you’re coming from and if you have any particular links or resources that you want to share with other attendees, please do so through that chat section rather than the QA. Leave the QA for questions for the panelists, use the chat section for anything more general that way. Please make sure that you set your visibility to everyone. The default is for host and panelists, but if you switch it to everyone that makes sure that everyone through the chat will be able to know who you are, see your questions, and hopefully be able to spark some inspiration between everyone there as well. My dearest colleague Jenny Gordon is going to be monitoring the chat and will be able to help anybody that is an attendee today in conversation through that same chat. Make sure that everybody is set to everyone. Send those questions through the chat that you have throughout or any type of comment or remark that you have, and Jenny will take care of you throughout the presentation.

Now, for the fun things, today’s conversation is about strategies for addressing the use of AI in classrooms. How do you acknowledge the use of technology for the benefits of students without effectively giving them a cheat code? Along the way, we’ll give examples of assignments and assessments that effectively measure acquired knowledge, authentically assessing skills in motion, and how you can leverage technology tools to help students develop in demand skills. Joining us today are two, like I said, spectacular humans that I’m really excited to introduce here today.

First we have Jeff Przybylo, a professor of Communication Arts at Harper Community College and the Director of Speech and Debate Program. And Joe Kennedy, the Coordinator of Instructional Design Services at Concordia College, where he’s also an intermittent faculty member for the math and education departments. Without any further ado, I’m going to hand it over to our panelists to introduce themselves just a bit more and go into detail. With the limited time that they have to introduce themselves, please do yourselves a favor and look these two individuals that they are incredible and very, very fun to have here with us today. Jeff, why don’t you kick it off, go first and then Joe hop in and piggyback from there.

Jeff Przybylo:

Boy, the expectations you’ve set up are incredible. I hope we can live up to it. My name’s Jeff Przybylo. I’ve been at Harper Community College, it’s in the northwest suburbs of Chicago. Just started my 28th year. I teach the basic speech course as well as intercultural communication and small group and leadership, and then I coach the speech and debate team and both GoReact as well as our sub subject here, AI have come into play in all of what I do. And this week since we last talked, Peter, in my role as department chair has also come some AI stuff, so it’s coming. It’s here.

Pete Morgan:

Great.

Joseph Kennedy:

I’m the instructional designer at Concordia College, and in that role we’ve been working with GoReact as a college for 10 years, so from the point of view of academic technology, I’m the administrator and help professors use the tool, and then also as the Coordinator of Instructional Design, I’m one of the people who’s helped guiding the college’s response to generative artificial intelligence. We are a very small college, so fewer than 2,000 students we’re an ELCA private school. I, in addition to serving as instructional designer, teach in the math and education departments every year or two and am a former speech and debate coach at the college. In all three of those roles actually, I’ve used the tool of GoReact as both an instructor and as an instructional designer grappling with the issue of generative AI and how it can be used to help students rather than be considered as, again, I think you said it best, Peter, a cheat code.

Pete Morgan:

Perfect. Well, I appreciate you both introducing yourselves. I think to kick things off here, I’d like to keep the batting order, if you will, in line. So maybe Jeff, I’ll point this very first question towards you and then Joe, direct it to you as well to elaborate further. The first thing that we really want to focus on throughout this webinar is talking AI strategy, different pieces that are available there, and then we’ll couple all of that towards the end in the second half here with authentic assessment strategies. So maybe Jeff, let’s start with you. When did you first realize that students were utilizing AI within their coursework? I mean, and maybe even a better question, what came first? Did you start to notice students utilizing AI and pick it up on your own, or did you proactively introduce it to students knowing that it’s coming down the pipe?

Jeff Przybylo:

That’s a big question. Before I answer that, I want to give a shout out to Chris. I see Chris Dobson in here. He’s one of our main guys down in our instructional design center, so thanks for being here, Chris. Truthfully, AI is one of these things I’ve kind of always ignored. For years you keep hearing it’s coming, it’s coming. I’m like, yeah, yeah, yeah, we’re fine. I’ve always been conscious, because I teach a lot of online. I was one of the first among many teaching the basic speech course online, so how do you keep students from borrowing work from somewhere else, or some people might call it cheating or giving the speech themselves and making sure that it’s not someone else that’s there. So it’s kind of always been in the back of my head, these ideas that I think everyone right now, since November, December of this year, have all of a sudden been smacked in the face with. So it wasn’t a shock.

It wasn’t until November when it started getting traction in the news when I went, oh, maybe it’s time to stop ignoring this and thinking that it’s not going to come, but instead maybe start to think about it. And then all of a sudden all these news reports happened where news reporters were letting ChatGPT or some other thing write their news story and then delivering it on the news, and it was like every single newscast was running that same exact story.

And then went back to school and we were coaching the speech and debate team. One day we were having a workshop and one of our students said, “Hey, Jeff, come here. Look what I just did.” And he had ChatGPT write a 10 minute, three point speech to entertain speech with all these criteria, and we read it and I was like, “it’s really not that funny, but wow, look what you just did.” And that’s what really opened my eyes. I was like, oh my gosh, maybe we need to start paying a little more attention to this. And then told him, “no, you can’t deliver that speech.” He wanted to. So now I’m aware. So now I’m kind of in this place where I’m being an observer. I know Joe’s college is being a little more active and I’m here watching colleges like Joe and listening to conversations like this to think about what is it that we need to be doing and concentrating on.

Pete Morgan:

Great. What about you, Joe? When did you start to notice it?

Joseph Kennedy:

Well, I started to notice it in my role as an instructional designer before I did as an instructor, and it started with a professor sending me a news story in December saying, “Do I need to be concerned about this?” And it was a fairly alarmist news story, and it immediately made me think systemically of the calculator wars when graphing calculators became affordable to students. But the big difference there was, for 10 years, mathematics faculty were able to grope with this and grapple the issues because the rollout was so gradual. But generative AI basically in November of last year, boom, there it was, and everybody had access to it.

So we started to look at it as a college, and then I started to talk to professors, and professors noticed it because at the beginning of the spring semester to the end of the spring semester, they saw a remarkable jump in the quality of written work from some of their students. And to be fair to the students, this was not in writing classes, but it was in classes where the professors routinely use written work as the main mechanism for understanding what a student knows and can do. I’m introducing it to my students, and in the context of math and science, most of our students have never thought about it that way. So we at my college have been introducing to them and giving them ways that they could use it to help them.

Pete Morgan:

Yeah. So Joe, tell me this, in introducing it that way, setting up the expectation, if you will, on best practice ways to utilize the AI, what is the expectation from your perspective for your students utilizing AI in your courses?

Joseph Kennedy:

All right, so to keep this short, Jeff can give me time signals if I’m getting too long. The first is to remind the students of the general expectation for academic integrity within the culture of the institution. My undergraduate institution had a very strict honor code, the kind you read about on the news where students all fail a class because someone cheated and no one reported it. The college I am at does not have a code like that. They have more of a culture, and so we have to fit it into that, reminding students that the whole point of an assessment is to know what the student can do. And so it’s incumbent upon the student to represent themselves and their work and their thoughts, but it’s also incumbent upon the professor to explain to the student what is being assessed. So if we are assessing students’ understanding of the Napoleonic Wars and how it set up World War I, then maybe we don’t care about the quality of their writing. We should explain that to them that what we’re looking for.

So the college came out with a statement that reminded students of that. It also reminded students that because generative AI tools work by essentially scraping billions of data from the web and amalgamating it, that if a student is to use that, they are in essence plagiarizing if they don’t give credit. So giving them something specific they can latch onto and then explaining it in terms of what can they do about it is say, “I’m quoting this source, it is ChatGPT. I used it on May 15th.” So that’s the overall expectations.

Pete Morgan:

Yeah, so it almost sounds like that blends into as well some of the strategies. Setting the expectation is a part of a strategy to know AI will be leveraged in some way or another, probably more often than not as we go forward. What are some of the other things that you can think of strategy wise here, Joe, to create good assessment opportunities where we know AI will be leveraged, but we also are requiring students to demonstrate their learning and knowledge as authentically as possible.

Joseph Kennedy:

Well, one of the things that I do is I show them what they can do. So in a mathematics class, one of the problems is when a student doesn’t understand a concept, do they go in for tutoring? Do they go in for office hours? How does a professor retest a student if they’ve done poorly? And so I show the students with AI up on the board, we go into a live session and I show them how to prompt it to give them sample questions that get to the heart of what they’re not doing well and teach them how to be a prompt engineer for this very specific purpose. And then the students will take the problem, solve it, feed the answer back in, and then they turn all that in. So they’re creating sample questions for themselves that will target the thing they don’t understand. And many students, that’s all they want. They don’t need a tutor to explain it to them. They just need to practice it again and be guided a little.

Pete Morgan:

Yeah, almost as if you can train the correct method in how to utilize AI, you can narrow the scope as to what things will be utilized because they know they’re doing it in best practice or in the best format possible. That’s a great thought. I know, Jeff, as well. To piggyback a little bit off of some of these topics, you have a personal discovery story in this same regard, how we set up activities, assignments, knowing that AI is going to be utilized, but keeping authentic assessment or knowledge check in focus here.

Jeff Przybylo:

Yeah, I stumbled upon something before this past November when this all kind of became really popular to talk about, and I call it a go reaction. So I’ve been using GoReact since the beginning, and typically I was recording speeches and we were evaluating each other, but then I started using it as a discussion, not a discussion prompt, but instead of a small essay, which I do a lot of in my teamwork class and minor intercultural communication class, I would give the students either a written prompt or a video from YouTube or a video from me where I ask them and then they use GoReact to not write, but to actually talk through the answer to my question. So instead of reading papers, I’m on GoReact listening to students talk through their answers, and I can hear whether it’s their authentic voice or I can see if they’re reading something that they pre-prepared on the screen, and I’ll encourage them to not pre-prepare too much, just to jot some notes down.

That’s been really fun to do. It was great going into this. Now that we’re kind of in this world, I’m starting to use it a lot more. A second really brief thing that I’ve been doing, and somebody said this and I just thought it was the greatest thing. The best technology that we have right now is pen and paper. So I’m doing a lot more inside of class where I’m asking them to write something, small things and also along with pen and paper, small groups. Put them in the small group discussions or dyads and give them something to talk about, things that I used to ask them to write about and then sit at the front of the room, listen to them. Talk about authentic, it’s forcing them to be authentic and real and not getting their thoughts from somewhere else. And that’s been very effective.

In fact, this past week I did it in my honors speech class for the first time, and the discussion was incredible. I gave them a question that I normally would have them write about, but I just saved the end of class, threw it out there in small groups and let them talk, and it was great. It was better. I mean, so AI has pushed, and I hope continues to push, and does push the rest of us to more creative ideas and teaching strategies instead of fearing this thing and running from it and making rules and telling everyone to stay off it. Personally, I haven’t addressed it in class. I know Joe’s college is doing that, but right now I’m afraid to bring it up actually.

Pete Morgan:

Interestingly enough, Jeff, you bring up so many things that are a part of even GoReact’s authentic assessment toolkit. Being able to, what some of the taglines are to outsmart AI or even if you’re not bringing it up as a topic, doing things to become more creative in how we approach assessment opportunities, how we approach great conversations and authentic thinking, skills development that way. There’s a lot of really cool things to look at that as well within GoReact’s AI toolkit, but really good points there. Joe, you were just about to hop in and I interrupted you.

Joseph Kennedy:

Oh, it’s okay. I think it’s interesting that Jeff says, “Oh, I’m not addressing this.” But you really are. The huge positive of ChatGPT and generative AI, and it is very hard. I want to acknowledge that for faculty members, it’s very hard to deal with this. We’re coming off the pandemic. Generative AI is as disruptive to assessment as the pandemic was to the entire educational experience. And so we’re saying to faculty members, you just dealt with something that profoundly changed everything about your career. Congratulations, you have to do it again. But what Jeff is saying is, his solution is to actually go back to what the point of assessment is, and Jeff is using this potential cheating tool, this potential roadblock instead as a chance to reexamine how he knows what his students know and can do. And so the students are coming out better for it. So I would say, Jeff, you may not be addressing how do I prompt engineer in ChatGPT, but you are addressing what’s at the heart of it to begin with.

Jeff Przybylo:

Yeah, I think I agree with you and I think what’s really happening is I’m just not saying it out loud in front of them and not yet teaching them how to use it effectively, because I think that’s where we need to go. We can’t just be policing it, we have to teach. I’m not ready yet.

Joseph Kennedy:

If it helps the first statement… So our college learned, you can’t have a policy on generative AI because a policy at a college takes months to craft and then it has to go before the faculty. And so that’s another couple months, but you can have a statement and the statement our college said when you did the TLDR was, there’s this thing, it’s powerful. It could be really cool. It’s new and we haven’t fully grappled with it yet, but we do know that there’s some ways that it could really be bad. Please don’t use it that way, and then let’s explore how to use it together.

Pete Morgan:

So in that thought process, I mean, thinking about taking a step back and becoming a little bit more creative in what we can do, to understand those points that you just mentioned, Joe, knowing that we can do really incredible things here if we understand it together and correctly. Perhaps there are a couple other examples on your minds, either of you, around different assignments or assessment activities that can effectively measure acquired knowledge whether that be in the use of AI or not. What else is on the top of your mind for examples of those activities? Maybe pushing some inspiration to those that are listening today on what we can do to take that step back and be a little bit more methodical when we approach this topic?

Jeff Przybylo:

I can continue to do what I’ve always done, progressive assignments, and so I’m prepared. So our students have to do an informative speech. I don’t just assign an informative speech and then let them go up and do it. First, there is a brainstorming session with their group. Then they have a topic proposal that answers a couple of questions. Then they have a simple outline, then they have a more detailed outline, then they deliver a speech. They would have to do a lot of work if they had ChatGPT write their informative speech and then have to reverse engineer all of those assignments. And if they want to do that, I’m okay with it, because they’ve analyzed, they’ve broken it down, they know the parts. It actually, you might learn more from reverse engineering a whole speech that someone else wrote down to the proposal level that’s there.

Joseph Kennedy:

Yeah, I really like that idea. And the nice thing about chunking and creating progressive assignments is, if Jeff has a student with great ideas who is super disorganized, it’s really easy then to say, “Well, here’s a very specific way that this tool can help you.” Because you can say to ChatGPT, “I’ve got these 13 different ideas, how could I organize them?” And at least it gets the students started. It can get them over that roadblock. So maybe they have a great brainstorm session and their initial proposal is great, but then when it comes time to turn in the first outline, Jeff’s looking at it like, this is an MKO mess. I think I see some glimmers of genius here, but I really got to look. That’s where the tool can then be brought in and build on what is already effective.

We have a faculty member who wants students to engage in online discussions. So Jeff, you’ve worked with forums online I’m sure, and there’s always that group of students that just never seem to say anything. One of the reasons the research tells us that this form of assessment doesn’t work is that the students are uncomfortable in that social sphere, but they can take their ideas, and this is what one of our religion professors is doing, encouraging students who feel that way to take their ideas and say to a generative AI tool, they could use Claude or Bard or ChatGPT, whatever one they’re using, this is what I want to say. I’m putting it in an online forum for a class. I’m worried about being disrespectful to other students. How could I phrase this differently? And then it gives them a suggestion. If they don’t like it, they can regenerate it and they can regenerate it, but at least they have a starting point to help them with that issue.

Pete Morgan:

Yeah, very interesting topic there. And also I’m seeing a surge of different comments on the same thread inside of the chat, but talking about ways that we can take what is traditionally a presentation model for things like research or data, all these pieces that obviously ChatGPT and Claude and all these other generative AI engines thrive off of. Because like you mentioned Joe, they’re grasping at billions of these data points to gather in and just produce or place that in front of us. There’s so many people that through the chat are talking about introducing not just those opportunities to yes, utilize AI and gather that research those data points, but utilizing video tools or in-class opportunities, really creative activities to share opinions on those data points, on the research that’s being done. The personal perhaps introspective look at what those things mean to the individual. Have either of you dabbled in something like that, of driving people to think inwardly about the data that’s being produced by AI and what has that done for your students or what types of outcomes has that produced?

Jeff Przybylo:

You got this one, Joe? He’s on mute.

Joseph Kennedy:

Yep. I tend to make a lot of random noises. One of the ways to help students is to make sure they understand none of generative AI has a corpus of knowledge. Generative AI doesn’t know anything. If you ask ChatGPT, “Who is the first President of the United States?” It says George Washington because out of all the data that it is analyzed in 99.9% of them, the phrase “first president United States” and “George Washington” are all together, but it doesn’t know.

So if you could convince 10 million people to go onto Reddit and Facebook and Wikipedia and so on and say that Abraham Lincoln was the first President of the United States, then the next version of ChatGPT, once it has read more current stuff online, will sometimes tell a student that Abraham Lincoln is the first President of the United States. Because again, there’s no corpus of knowledge. It is predictive. But once students know that, it can really help them understand what it can and cannot do. I know that’s not a very specific assignment if that’s what you were asking for, and I’m really restraining myself because much like Jeff, I would love to talk about 20 ways that GoReact is a great tool for this. But I know that’s coming up later.

Pete Morgan:

Yeah, exactly. We can leave that for the future webinar sessions or for people to look up our AI strategy in the AI toolkit, but really great answer there, Joe, personally. Just because what we want to do as well is spark the inspiration for others to develop what they’re going to do on their own right. Examples are great, what you’ve seen in success is wonderful, but everyone’s going to have to do these things on their own and find what works inside of their courses. So I like that answer because it leaves it open just enough for people to draw in some of their own creativity and thought process there. Really well put.

So maybe if we utilize that, just that transition and say, okay, we know AI is here, there’s lots of great ways that we can utilize it. There’s lots of things that can help us think more creatively on how to inspire students to do work differently, to think differently, to apply their skills in another way. Perhaps that’s a perfect segue into talking about authentic assessments and authentic assignments. How do we create these things or what are some of the key components there? So maybe I’ll open up, Jeff, maybe I’ll point this to you first since Joe has had some fantastic detail in the last little bit of the conversation. From your perspective, what are some of the key components to making an assignment authentic, some sort of activity or assignment authentic? Then we’ll go into assessment afterwards, but maybe touch on the assignment first if you would.

Jeff Przybylo:

Yeah, make sure your assignments have both parts. If you were asking a question about history, what year was this? Or who did what? That’s fine, but then following up that question with why this happened or that happened? Or how did you feel about this? Or when you read this, did this remind you of anything from your life? Really forcing students into taking the information, no matter where they got it, whether they pulled it out of the textbook or got it from AI, I guess it doesn’t matter. But putting that application level in there, and I’ve been going through all of my discussion stuff in my online classes and making sure I’m adding that part to it.

And I’ve been grading already and really starting to see, and I added a question in their introduction essay this week that said, what are your strengths as a student? And they just started talking about themselves and I’m like, this is really neat. So I’m starting to put more personal stuff in there and I don’t think that’s a genius idea. I hope that we’re all doing that anyways, but I think in light of everything we’re talking about here, it’s become more important. We really have to get down to that personal level and application analysis level in our assignments.

Pete Morgan:

So if we utilize an assignment in that way, Jeff, how does that prompt you to create or develop the most crucial components of an authentic assessment? What are those pieces and how does that couple together?

Jeff Przybylo:

Boy, I’m not 100% sure what you’re asking me. I sound like one of my students. You mean like creating a rubric or giving them feedback?

Pete Morgan:

Well, maybe that’s a perfect question here. I mean, I know that in the previous discussions we’ve had before the webinar, the idea of really what is authentic assessment is a question. There’s a lot of different meanings that that can possibly have. So maybe Jeff, the right question here is from your perspective, what is an authentic assessment? And how do we take an authentic assignment, something that carries both of those parts you just mentioned and allows you to assess those skills authentically? That’s a lot of authentic wording.

Jeff Przybylo:

I got it. Be authentic back. So if I’m creating an assignment where they have to be authentic coming at me, then I need to go be authentic coming back. Now, I love GoReact’s rubrics, I’ve been using them, I lean on them, I’ve really been developing them. My rubrics are not just superior thoroughfare I’ve written very formative descriptions. So my rubric is also teaching, but more importantly, I use GoReact to do what we’re doing right here. So when I watch an informative speech online, which takes a lot of time and your butt falls asleep, I’ve got 70 online students this semester, I watch the entire introduction and then I stopped their speech. I don’t watch the rest of it. I hit the video and I talk to them, I coach them. I spend two or three minutes saying, “All right, your attention getter was this. Your central idea looked good. You really need to work on your justification. I love your delivery style. Your setting is great.”

I mean, I just spill everything out of my head. I’ve been a speech coach for almost 40 years, so we’re used to doing that with our speech team and debate students, but we never really do it with our classroom students. I mean, we do a little bit afterwards. So they’re getting coached in the middle and then I say, “I’m going to watch the rest of your speech and then I’m going to visit with you again at the end.” I watch the rest of it. I jot down some notes, I have good shorthand techniques and at the end of the speech I say, give my opinion. Give them compliment, do the compliment sandwich, tell them all the good things, tell them what they could be working on, leave them with something positive. And they’re not getting a rubric, they’re getting a rubric also, but they’re not just getting copy and pasted comments, which we’ve all done. If you’re not, you should do that because it saves a lot of time.

And when you do do that, you should personalize them. That’s another thing that I do. Even if I have something really neat that helps them with a certain part of their speech, I always personalize it, whether it’s as simple as using their name or adding a quick reference to what it is they’re talking about. Those kinds of things make your feedback authentic and like I said, the most authentic thing I’ve been doing lately is talking to them.

Pete Morgan:

Well, and what a great way to, I know this isn’t really the topic, but show how real you are as the person providing the assessment. That it’s not just plugged into ChatGPT and say, give me some empathetic response that somebody’s doing a great job and a few constructive things based on this prompt. That’s a great way to really show how real and genuine you are as the evaluator. Joe, I want to point it to you and especially from the instructional design perspective, what are some of the most crucial components of making an assignment authentic from your perspective?

Joseph Kennedy:

The assignment being authentic or the assessment? Because I-

Pete Morgan:

I’m going to go both ways. I want to start maybe with some of the characteristics of the design in the assignment and then same thing separately for the assessment, knowing that they’re two different parts here.

Joseph Kennedy:

I’ll give a shout out to Christopher. So apparently also someone from Harper College, he’s posted something in the chat about UDL guidelines and that’s really key for authentic assignments. The student has to be able to authentically show you what they can do and what they know. And so again, if it is a public speaking class, they have to give speeches, they have to orally present. If it’s a writing class, they have to write. If it’s a history class, unless you are teaching them how to write to be published in a history publication, maybe they don’t have to write. If it’s a business class, unless part of the focus is how to give a presentation to investors, maybe they don’t have to present. So part of an authentic assignment is finding a way where the students have multiple means of expression and when you respond to them, you’re going to be able to respond in a way that helps them.

Larger topic, that means you have to think about grades and what the grade means. That’s like 20 other hours of discussion, but it’s one of the reasons that I like GoReact is the way that Jeff uses GoReact and I envision using GoReact ism you can give students a way to orally present when maybe you’re used to using a paper, but you can still be efficient and authentic in your feedback. And that’s where the assessment part comes in is, if the only reason we’re assessing students is to figure out a grade to give them, then the assessment is inherently inauthentic to begin with, with a few exceptions.

If you’re a nursing student, you need to know the 50 most common drug interactions and you have to know them cold or people die. There’s not much feedback to give on that. You know it or you don’t. But in terms of giving a speech, crafting a paper, talking about historical imperative, solving math problems, there’s a lot of process discussion that has to go on and that’s what makes it authentic is, can the student demonstrate what they can do and know so that you are evaluating that rather than evaluating the form in which they did it? And can you as the instructor provide feedback that will take where they are and show them how to get better?

Pete Morgan:

Joe, I’m going to come back to that here in just a moment to talk about some of the key benefits that you see there. Before we jump into what will be obviously a great response from you since you’re so detailed that way. Jeff, I want to point back to you and jump on one of the topics that you were going on. You’re talking about utilizing GoReact as an assessment tool using video and feedback to speak to your students to provide that a little bit more personal touch. Are you utilizing that in any way from your student perspective to develop in-demand school skills personally? Are students prompted to use video responses within the GoReact feedback panel? What are some of the things that you’re doing there to leverage video and feedback from your perspective like you a little bit already explained, but also from the student learning perspective?

Jeff Przybylo:

Yeah, well, you read my mind. I was going to jump in and add that to what Joe was talking about. Absolutely. We all in some form or another do peer feedback and the old fashioned way in speech class is you’d hand out little half sheets of paper and they would take notes and then you’d sit in your office with a stapler and sort them out on the floor and hand them all back. GoReact has put an end to all of that. Up until about a year ago I was having students, I would give them a different rubric and they would watch, I put them in teams so they didn’t have to watch everyone in the class. So they’d have their team of seven, they watched six speeches and fill out a rubric. Pulled the rubrics out, also pulled out my impromptu speaking assignment.

So now what I have them do is I say, watch your teammate and I want you to write down the three things that they’re doing well and three things they can stand to improve on and then I want you to hit the camera and in one minute give me a two point speech with three sub points under each one as your feedback to your students. So it’s working both ways. They’re giving… If we’re talking about authentic evaluation, we’re also testing their impromptu speaking, so there’s a grade attached to that. So they’re doing, I think my perspective is they’re doing these better than they’ve ever done them before. They could just click those rubrics and not even be looking at anything, and there’s so many of them, how could I be keeping track?

The drawback is, this is taking a lot of time. The good thing is, and we don’t like to talk about the times two button, but you could watch some of these one minute impromptu speeches, you listen to the delivery for a couple seconds, hit the times two, I’d love GoReact to maybe go to times three because sometimes students are speaking so slowly that when you hit the times two, they’re still going too slow. So there’s little tricks like that to be able to watch these and I’m going to do this semester, make those groups a little smaller. Seven was a little big. I may go with six and five when I set those up in the next couple of weeks, but lots of fun stuff going on there and my feedback on my evaluations, there were quite a few students that said we really liked doing the oral feedback, which to me was oral feedback, but it was also my secret impromptu speech.

Pete Morgan:

Yeah, great point. And what a layered activity and opportunity to provide technology at the fingertips of students to utilize a great tool. Really, really great example. Joe, let’s use that to jump right back to where you had left off previously. What I would think everybody here would love to know is top maybe two to three benefits of utilizing video for authentic assessment when it comes to courses or disciplines that are not as writing heavy, things in STEM studies, for example. Give me your thoughts on that.

Joseph Kennedy:

Well, it’s actually very similar to what Jeff does. The first is it provides multiple means of expression. And this is a huge problem for math students. Students who hate math, and there are a lot of them, the only higher percentage of students who don’t want to give a speech in public. So students who hate math, they hate math because they feel they have to use this foreign language and these symbols and equations and a tool like GoReact makes it very easy for them to just get up and talk. And I tell the students, you can do an epic rap battle with your friend. You can create a dramatic monologue. You can just talk to the camera and explain what you’re doing. And some students, they just aim the camera at their hand and they work out a problem and they speak aloud what they’re doing, and that gives them a chance to give feedback.

It also prepares students for life after college because no one ever walks up to them in a dark alley and is like, “I’m going to shank you if you can’t factor this quadratic equation.” But what does happen is they have to stand up at a school board and explain how the data that they’re seeing makes them think that money is not being spent well, or they have to argue with the business about the way numbers are being used and they have to interpret data. And being able to do that face-to-face takes up all the classroom time you have. Having them do presentations like that in GoReact allows you to give them feedback without taking up all the class time because you can do it on your own. But most important, it allows for self-reflection and affirmation. One of the key things I like about GoReact is I can ask students to explain a math problem and then watch the video and comment on the video about, do you think you explained that well? Was it complete? Would a 14 year old understand what you are saying? Would a business person understand what you’re saying?

And I will also provide that feedback, but they don’t see my feedback until they’ve already done theirs. And that gives them a chance to not only self-reflect, but also to see if their self-reflection is in line with the instructors, the person they think is an authority. And that can be very affirming to students when they’re like, oh, I kind of thought I was good there and then the instructor told me I was amazing. That can really help students get better in their ability to understand things in the math and science fields.

Pete Morgan:

Self-reflection and affirmation. I love that and think how powerful, not only just to reintroduce and make sure that self-reflection is such an important topic, making sure that we’re learning from ourselves, but seeing and drawing those connections from, I think that was good and being told it was really good, what a confidence booster. That’s fantastic. Well, everyone, we’ve reached the end of our discussion time and there’s obviously so much more that could be discussed here that will possibly come down the pipe for us later on, but I think this is a great time to transition into our QA session and make sure that we can bring in some of the questions from our audience.

Now, leave this as my invitation. We already have one question that’s there that I’ll open up, but while we pose this very first question and get some great responses from our panelists here, open up the QA, jot down those thoughts and responses, anything that would be top of mind for you to ask our panelists based on the discussion or based on the inspiration, the lightning sparks if you will, that you’ve received in your own spheres while we’ve been listening in.

So the very first one that’s come through the QA, Jeff, I’m going to point towards you or our attendee pointed towards you. This is from John Beal and he asks you, “How do you answer the question from the critics when using progressive assignments, what keeps students from asking AI to write an outline and then plugging that outline in to write the actual speech for them?”

Jeff Przybylo:

Nothing, I suppose. That’s my cute answer. I think it’s that old adage if you’re watching their work throughout a semester, you just have that feeling. You can tell when all of a sudden something is way too good or following things too closely. That’s a great question. I think we need to struggle with it. I do think, and it kind of fits here, a thought that’s been occurring to me while we’ve been talking is that if we really want to talk about rules and strategies, we’ve got to use this thing. If anyone on this panel doesn’t have a ChatGPT account, you need to hang up and go get one and use it because you’ll start to recognize some patterns and you’ll start to see how it likes to lay things out. And that’s really helped me to recognize, and I’ve recognized a couple.

As an example, on a lot of questions, you’ll ask GPT a question, it gives you a paragraph answer, and then it gives you nine bullet points to support it, and then a nice summary. I’ve seen that come through a couple of times and I’m like, listen, you’re a freshman at a community college, you ain’t that brilliant. Especially looking at your work that you did previously. I haven’t said anything to anyone yet, but yeah, I suppose they can put it in. What’s going to stop us from doing it here? “Hey, give me five great questions to ask in a seminar about authentic assessment.”

Pete Morgan:

Yeah, and I think that that trend or the topic of the predictable trend. It’s very interesting because as I’ve utilized ChatGPT in my every day and trying to further understand it, not only am I able to recognize some of those same patterns, but I’m also starting to recognize that simple technology, something like some of the tools that are available in GoReact, provide an authentic response, or I should say opportunity to respond authentically to support or combat or argue what ChatGPT is fitting out there. So whether students are going to leverage it or not, I think we all know it’s going to happen, but utilizing some of that predictability that it will be leveraged in some format and offering a coupled or a conjunctive assignment or activity to say, now tell me about it. Why does it matter to you? Tell me why it’s personal. Give me a story, something that way. I’ve found that there are really simple answers to providing that opinion state or the personal touch to it. Great point.

Jeff Przybylo:

Really quick, direct answer to his question, extemporaneous delivery.

Joseph Kennedy:

Yes.

Jeff Przybylo:

Yes, let ChatGPT take your outline and write a speech, but they still got to get up there and read it. And I do check their note cards. Even in an online class, they hold up all five of their note cards and they show me their notes without cutting or editing or anything like that. So if you’re good at taking a manuscript and turning it into notes on note cards and doing it extemporaneously, then you’ve learned something, I suppose.

Joseph Kennedy:

It’s funny-

Pete Morgan:

Joe, you were going to jump in there. Let’s hear it.

Joseph Kennedy:

Oh, I was going to say, it’s funny because that was the very first thing I argued for my faculty when they started getting concerned in February and they said, but can’t a student just read to the camera what ChatGPT wrote? And I said, look, and Jeff has worked with them as well. We worked with some of the most accomplished public speakers in the country. Jeff’s students won a national championship this past year, giving a shout out there. He gets to see the best of them. Very few students are anywhere close to that, and you can tell if a student is reading their own stuff and is nervous or if they are reading someone else’s. You just have a good sense.

Pete Morgan:

Great point. I have another one that’s come in through the QA and I’ll open this up to the both of you as well. So whoever wants to stab at it, please feel free. But the precursor to this, the description is, “AI is here to stay. It’s not going anywhere. It’s already taken a foothold in business, which means it’s going to be a necessary tool to learn for students. My read of the chat identifies how we can teach students the ethical and responsible use of AI as a main concern. But for our panelists, are there any initiatives or guidelines that address this, the ethical use of AI?” Is there anything that is out there that you’ve noticed or recognized that could be shared with everyone?

Jeff Przybylo:

I think Joe’s probably run across some stuff.

Joseph Kennedy:

Yes, but what keeps getting lost in all of the various things that I’ve looked at, all the models, what I’m not seeing people come out and say explicitly is, AI, like GoReact, like a graphing calculator, like pencil and paper, is a tool. And if we tell students how to use a tool and we don’t tell them why they’re using it, we have already lost the battle about ethical use. They have to know why they’re using this tool versus another one in addition to knowing how to use it. And I know it sounds naive, but the data backs me up. Faculty members who explain to a student why they have assigned something, what the point of the assignment is, and how it relates to the course as a whole, always have fewer issues with academic integrity violations than faculty who just assign things and assume that by fiat or authority, the students will do it the way they’re told.

Pete Morgan:

Joe, it almost sounds like we’re going to need an article from you to plot out the ethical delivery of AI in this world. Does that sound right?

Joseph Kennedy:

Put it on the list.

Pete Morgan:

That’s fantastic. Well, everyone, we are coming towards the end of this. I think we’ll leave it maybe just one more minute. If there’s any last minute questions for our panelists here that you want to place through the QA, please feel free to. Before that or while you’re writing that out, maybe I’ll just say to everyone early on, thank you so much for being here. This has been really fantastic, such an insightful conversation and a special thank you to Jeff and Joe for the discussion. I patted them up in the beginning. You heard me talk about how marvelous they are and they definitely didn’t disappoint, so we really appreciate the two of you being here with us.

There’s a lot of great information here, so many different takeaways and inspirations that we can draw into our own use cases, into our own worlds and know that all of this will be shareable and will be able to come out for those that need it. Anybody that you’d like to share it with, this recording will be shared out through the email and the opportunity we had there. Are there any last minute questions through the QA that we can handle before we close up or before we offer a couple last minute thoughts here?

Well, for everyone here, maybe just a final thought. We all know that AI is here and like we’ve heard, like we’ve discussed, it’s here to stay, but there are so many opportunities that we have to authentically assess students in their skill, taking that one step back, being more creative in how we deliver these different, perhaps more traditional learning methods in today’s day and age. There’s so much that we can do to provide authentic assignment design that fuels true authentic assessment. This all will lead into so many other discussions down the road, one of which towards the end of September, GoReact will have another webinar talking about authentic assessment that will piggyback on a lot of these ideas.

But I think it’s really inspiring to know that not only do we have the tools and technologies at our disposal to outsmart AI, but also as it becomes more of a mainstream topic to work hand in hand with those to make sure that our students are utilizing real world application and tools that will be at their disposal the rest of their lives to not just do great work, but truly understand it and develop the skills to use it correctly. Thank you to all of our attendees for joining. This has been a fantastic presentation. We hope to all see you in our future webinars. Remember for the end of September coming at you, and we hope that you have a great rest of your week. Have a wonderful day everyone, and we really appreciate you for being here. Bye bye.

Joseph Kennedy:

Thanks, everyone.