Learning on Both Sides of GoReact: How Students’ Feedback to Peers Improves Their Performance

Thank you for joining us, everyone. Welcome. Thank you for being here at ReAction. My name is Tola Oladipupo, and I am looking forward to today’s session.

J.D. Schramm is joining us from the University of California Davis for the session, “Learning on Both sides of Korea, How Students Feedback to Peers Improves their Performance.”

As we get started, we would love if you would introduce yourself in the chat, feel welcome to share any resources or additional thoughts. Please make sure that your message settings are set to everyone to share with the group.

If you have a question for our presenter, you can use the Q&A window. If you see a question there that you would also like answered, you can click the thumbs up button to upvote it. We will answer as many questions as we can during our time today. I’m now excited to turn the time over to J.D.

Tola, thank you so much. And to the folks who are joining on the halfway through the second day of the ReAction conference, welcome. Before I jump in with my slides, I thought it was useful to tell a little bit of my journey with GoReact.

We did the research–

and I think I started using GoReact in 2012 or 2013.

So I’ve been a long time user, first time caller. No, longtime user, first time presenter. And my journey has been interesting with GoReact. So I teach communication skills.

I’m going to be using examples today from my class at UC Davis, as Tola mentioned. But I’ve also used GoReact with my students at Stanford and Columbia and USC, and in different ways. But everything that I teach is somehow focused around communication, quite often, presentation skills.

And I’ve listened to a lot of the sessions over the last couple of days.

Boy, if anyone in here was part of the high school and college debate and forensics group yesterday, man, hats off to you. That was my first job out of college. And I was amazed thinking, wow, what could I have done as a debate coach or forensics coach if I had GoReact back in the day?

But the biggest thing that brought me to GoReact is I love to give feedback and hate to give grades.

In fact, one of my colleagues at USC said I teach for free.

They pay me to do all the grading. And that’s kind of how I feel as an educator.

So I enjoy giving feedback and watching students develop in their role as a communicator and as a speaker. But a 10-minute video presentation from a student, which was often my midterm at UC Davis and also at Stanford can take me 30 or 45 minutes to grade.

And if I’ve got 48 or 50 students or 70 students at times, like that’s a huge chunk of time. But the 54th student I grade deserves the same level of feedback as the first student I grade. And so I was always looking for shortcuts, anything that would make it faster.

And what I used to do, we had a system at Stanford where if you called my phone on my desk, which was rarely ever used, if you called my phone on my desk and left me a message, it converted it into an audio file and sent it to me by email, which was an awesome system because who sits at their desk anymore and who notices the little red blinking light that you have a voicemail?

So I love the system. And I found that I could hack that system and give students–

I could finish grading a presentation, call myself on my–

use my cell phone, call my desk phone, and get a two-minute, three-minute audio file that captured everything that I wanted them to know about that presentation.

So it was a cute workaround, and the students liked getting the audio file.

But then I had to attach the grades to the email and send the email out.

And it wasn’t handy.

I saw GoReact at the Management Communication Association Conference several years ago, and I saw the light bulb went off. I thought, oh my gosh, I cannot only be more efficient with my feedback–

with my time, but more effective with my feedback.

And so that’s what brought me into the GoReact family. And I originally only used it with Exec Ed students. I didn’t use it for students who were taking the class for a grade.

Then I started using it for me to give feedback, then I ultimately used it for peers to give feedback to one another. And now I’ll show you the full use of it. And I know there’s features within GoReact that I don’t use.

But the learning that I had, the experience that I had was for years, I had been saying that learning occurs in a communications classroom on both sides of the microscope. You both learn by being observed and getting feedback from your peers and your professor, but you also learn by doing the observing.

And GoReact, rather than filling out a form of peer feedback, GoReact allowed me to bring up the game of peers giving one another feedback, and bring up my own game of providing feedback. And I saw that students learned both in the receiving of that feedback and in the writing and providing of that feedback. And that’s what I want to talk about briefly in the half hour that we have today.

So I found that performance improves for the speaker. They get timestamped moments of would have been great to have eye contact at this moment. Would have been super if you had not tripped over that word, or looked back at the slide and given us your backside.

So those are all time stamped within. But then you got to be really careful, because that makes every comment look like it has equal weight. And the fact that you did not look at the camera or look at the audience at that moment may not be as important as the fact that you didn’t have a clear thesis, or you had no ask at the end of the pitch.

So the synthesis at the end allows me to really capture what was the most important piece that students needed to know, and one of the speakers talked about yesterday of all things that I saw, here are the three best things, here are the three things you need to work at, here’s the single lesson I want you to know. Like that structure as a possibility of how to give that feedback.

But additionally, I saw that for the reviewer, it improved their performance. As I started looking at what peers wrote to each other, they were often commenting about what they most needed to work on. So if I had a student who was struggling with filler words–

ums, uhs, you knows, likes, they would be the ones who would comment on filler words.

I had a student who was struggling with non-verbal communication, they would often be the ones who would comment on that’s a great place to move, or those were wonderful gestures, or did you think about using a gesture there? So the reviewers began to see patterns in their peers that they also face.

And I’ve got–

I’ll show you in a moment what it looks like in my classroom, if I’ve got four speakers up there, but 12 students writing feedback on it, I can also really see some patterns that are showing up and comment upon it migrating.

And at times, they saw stuff that I didn’t see. At times they saw stuff that I didn’t agree with. But I had that data in front of me so that I could show what that looked like.

So let’s look at this in the context of GoReact.

Tola, please interrupt me if my slides aren’t visible or my GoReact isn’t visible.

I just want to make sure I’ve got audio set here. Yeah.

So let’s look at this inside of GoReact. So this is what GoReact looks like the way I run it in my classroom.

I think those of you who are familiar with using it, you know you can slide this across if I want to see the image bigger.

And I also always have access to the video files. But I prefer having it all in front of me.

I did not learn until this quarter that I could assign the students to drop their slides in and match the slides up, so that I have the slide in large view.

I have the students in whatever view that I want them. And I’ve seen all these presentations. So it’s not the first time I’m seeing it. And then I can turn off or on the comments graph. From now I will just leave that on there.

The other thing that makes me efficient is the little tags; eye contact, fillers, tentative language; maybe, perhaps, kind of, and low volume. And those are tags that I could put in there, but also the students could use it as they began to tag.

So let me just show you the beginning of this paired presentation by two gentlemen. The assignment was do a 10-minute persuasive presentation that convince–

a six-minute persuasive presentation that convinces us of something.

These boys took an interesting take on it. They said, can we choose what date and time we’re delivering? I said yeah, absolutely.

So they chose 2001, and they’re going to tell us about a business opportunity to invest in. You may need to turn your volume up so you hear them.

[VIDEO PLAYBACK]

As you know, we are Steven and Tim. Together, we’re Mandalay Investment Group.

And now we want to talk to you about the new year, 2001, the official start of the new Millennium. We have an unbelievable investment opportunity for you that is perfect for this occasion.

We can all admit last year was pretty rough.

We saw the NASDAQ down.

In the fourth quarter alone, we saw the implosion of pets.com,

etoys.com, and go.com.

But wouldn’t you want your money in a company like this, your stocks?

While the NASDAQ is down 39%, this company is up 91%.

How do they do it?

Well, by providing products for the 21st century, by utilizing cutting edge technology, and by state of the art accounting methods.

I’m talking about a company that Fortune Magazine named top innovative company five years in a row. I’m talking about Enron.

[END VIDEO PLAYBACK]

I had to let them go long enough for you to see that they were talking about Enron. Now what’s interesting about this paired presentation, on this particular assignment, I didn’t have peers evaluate. I only asked Timothy and Stephen to go back in and watch their video, which I do in almost all of my classes, and to write their peer feedback.

And when I went into grade, I could see what Timothy and Stephen had already said, I could see where they dropped the slides in, and I could comment upon their comments. One of the speakers yesterday said one of the things we want to be able to do is enable students to evaluate their own work after they get out of our classes.

Now this was a masters of accounting program, and I knew that these two guys were going to be working for major accounting firms. And so my comments were often about what it was that I wanted them to think about when they did presentations like this on the job.

And so they get better with my feedback, and they get better by watching themselves and seeing their feedback. Additionally, as we went deeper into the course, I would also have their peers comment upon their work. Now with this, and I’ll show you another example of what that looks like, I do not have everybody in the class comment upon everyone.

So this particular class, I had 54 students. I think they were in nine teams. And so while team 3 was presenting, maybe teams 1 and 2 would comment. While team 9 was presenting, maybe team 7 and 8 would comment.

But they would not come in real-time. I know GoReact has that possibility. I don’t want my students to have their laptops open while their peers are presenting. So they’re assigned, and they know up front, they watch all eight other teams, but they know they have to go in and watch team 7 and team 8 and write feedback.

And so they’re paying more attention because they know they’re going to watch those videos and add their peer feedback.

So in this particular presentation, there were 118 comments that were made.

And you can see where the students made comments–

good opening, shock factor. Great opening, very engaging.

Good protection. A bit stuff, could have used more movement.

But they are rewatching two of the eight other teams and writing their feedback in there. And that is where I began to really see the peers picking up on what it was that they were also working on, and what it was that they needed to consider as they did that.

Now I also want to acknowledge–

kind of play this without the ball game, if I can talk over it. I don’t know how much you work with aspiring CPAs, but these slides and this concept of looking at a global issue like climate change, and being able to talk about what CPAs can do about that massive of a topic, which is really impressive for me to see what the students were doing and how their peers were able to comment upon it.

I also recognized at this point that I had another tool available to me that I didn’t realize was there. I was able to open my final presentations up to administrators, so that they could see first of all, the documented improvement, one of the skills that the AACSB, which accredits, accounting programs, Masters of Accounting Programs. They need to demonstrate to the administration that students writing and speaking got better.

This is a great tool to let my administrators see the value of what it was that we teach the value of the GoReact software, which actually at UC Davis, they didn’t pay for it. It was like a textbook that the students picked up the cost of.

But they really got a glimpse into my classroom without sitting in the back of the room making the students nervous, and they could completely with integrity say yep, students got better in their public speaking and in their writing abilities by what I showed them.

And I knew that I was getting faster and better with what it was that I was–

with how quickly I could get grading done and back to the students. I aslo had more data points and they were all right in front of me because everything was inside of GoReact. Slides were there, the video was there, the peer feedback was there. My comments were there.

And as somebody noted yesterday, I could turn on or turn off whether they saw my graded comments until I wanted them to be able to see that. But a light bulb also went off that this was the first time that I realized I could click here on comments, and I could actually get performance indicators.

Anariddha put 17 comments in, Brianna put 10, Pin-Jung put 13.

Ziyan only did two.

Hark only did five. Mita only did five. And I had one more data point about participation by the volume of comments that they made. Now one caveat with this at a student who only did one, and she was one of my top students. And I was stunned that she only did one comment.

And I went to look at her comment, and she had actually written out notes and typed them up for each speaker, and must have done that within the presentation. I didn’t realize she had her laptop open, or she wrote them out and then typed them up later. And she pasted one comment, but it was very rich. So I acknowledged her for that.

But some of the students had one or two comments. They just thought yep, you look nice, pretty slide.

Could be louder. I’m like, yeah, that doesn’t add value to your peers, and that doesn’t add value to yourself. So I was able to use that as a participation point.

But the last piece that I want to comment upon–

oh this is great, we have about 10 minutes left for Q&A, which I welcome, is remember I said earlier that I started all of this because I wanted to provide an audio or a video file that students knew how I felt about their work?

Well, and also that I don’t want look too much at the slide to be at the same weight as didn’t have a clear thesis statement. And so the last piece that I’ll show you–

and by the way, I do have permission from all of the students here to show these.

This is another team’s–

final team presentation. Once I get through all the comments–

now here there were 100 comments that were made, including my own.

I use the video function and I literally create a video file for my students right after I’ve done graded, after I finished grading it so that they know what I think is most important. I’ll just give you an example of how this starts out.

[VIDEO PLAYBACK]

Peers, and I could–

[END VIDEO PLAYBACK]

Let me go back to the beginning here.

[VIDEO PLAYBACK]

Yuhan, Bhutani, Kelly, Stephen, and Elizabeth, wow, I enjoyed your presentation last week when you delivered it, and I enjoyed it even more today, where I can see the comments from your peers, and I could be reminded of all that you put together for this presentation.

Your work as a team and individually was really, really strong, and I want to be sure you get that from me. It was clear how you opened up for Q&A, Brittany and Bhutani, and how you came back in for a final close.

From the slide deck, not just Luckin Coffee, but from roasting beans to cooking books, like little elements like that don’t happen by accident. You were strategic. Were thoughtful. You work together as a team.

There’s another piece that if we were all sitting together I would share with you. But I want to just acknowledge your choice of Yuhan and her effectiveness in that role.

Yuhan, you were in a tough position because the longer the presentation goes on, the less attentive that the audience is. But you were given a very specific role to bring it to our needs as future CPAs or future accountants, and you did that really nicely.

[END VIDEO PLAYBACK]

I’m going to stop with that feedback, but I wanted to let it play long enough that you saw the value of that synthesis message. She was probably one of the weaker speakers on the team, but they were talking about an integrity issue with a firm in China, and she was the best person to be able to synthesize that. And her growth, although she wasn’t as strong as the other four speakers, her growth was what I really wanted to be sure she knew.

And sometimes in those videos, I’m also telling them what didn’t go well. And being sure that they understand. But this was a very high grade, and I didn’t want them just to see the number and walk away, I wanted them to begin to see, begin to understand what it was that merited that number.

So overall, I see that GoReact has now a four-fold impact. Certainly helps the speakers, both what they see and their self-critique, it helps the observers build their skills, helps administrators know what’s going on in my classroom, much easier access, and it helps me be faster and quicker in what it is that I do.

So with that, I do want to offer my contact information. I’m going to open up for questions here. But I’ll leave this slide up for a little bit.

Please feel free to reach out to me by email or if you just go to my website jdschramm.com. That’s not a shameless plug for my book, which is featured there.

But you can just click on Contact Me and use GoReact in the subject line.

Be glad to send you a copy of the first chapter out of the book. And you can see if it’s useful to you in your teaching.

But mostly as the headline says, I just want to keep the conversation going.

So with that, what questions can I answer or comments do people have from the group? Wow, we went from 19 to 47 as I spoke. Welcome, everybody. Tola, how do we want to do questions here?

Absolutely. Yes, we have a Q&A window that you can see if you look at the bottom. There’s a Q&A button. If there’s any questions that you would like to ask, we invite you to share them there, and we will select from there. We don’t have any as of now, but we would love if you have any on your mind, go ahead and share those. And we would be happy to answer.

And if you prefer to ask it by speaking, I think we have the power to make them into a panelist for a moment so that they can talk.

OK.

Do I ever make private comments to one of the team members, how do you distinguish private and public comments?

Yes, I do. Because again, I’m trying to get through the grading quickly, I might just type a comment that says Steven, see me about storytelling.

Like if I need to have a conversation with him about an inappropriate story he told, a story that was too short or too–

I will do something like that, or there are times that in the midst of the grading, I’ll just pop an email off to somebody and say, hey, I watched your video again. I think we need to talk about some stuff. Let’s set up some office hours.

That’s usually how I do it.

It’s minimal how often I make private comments.

I kind of create a culture in the class that it’s a culture of rigorous feedback. And so they’re used to seeing my feedback for one another, especially on peered and team presentations.

How difficult was it for students to upload the videos to GoReact?

Oh. So OK, GoReact, huge compliments for how much better the system is after a decade of using it.

The upload is really quick. The way I do things, I have a video–

I have a TA that videos in class, and he puts it all on to just a simple card drive like this.

And then on his laptop, he just uploads the videos. And usually, he has made the videos available to students within 20, 30 minutes after class is over. The upload goes really fast if you’re on a wired connection or on a high speed wireless connection.

I know some teachers expect the students to film and upload themselves. I have one assignment that I do that is kind of an introduce yourself to the class and I have them film it on Zoom and upload it themselves. But I find it faster and quicker if my TA does it.

There is a new function, a new feature, which allows you to use YouTube videos.

And I have found that my students are usually more facile at creating something and posting it to YouTube. So if I have a student who misses class, for example, I’ll say OK, well, posted to YouTube, make it private so that not everybody is seeing your benchmark presentation. And then upload it to GoReact.

The piece that I offload back to the students is they need to take their slides, create them into a PDF, upload and timestamp them where the slides show up. And I make the students do that part themselves. And again, that just makes my grading go much, much faster.

Were the groups presenting live and someone recorded their speech into GoReact? We didn’t record it into GoReact, we recorded it as an MP4 file, and then uploaded all the MP4 files right after class. I did have one time where I had the students doing comments live on GoReact as the student presented, and it just felt like too many moving parts for me.

So I don’t use that feature, but I know other people do.

The advantage of doing that is you have less post-production. The disadvantage is the talks themselves don’t feel as authentic. It feels more like they’re speaking in a production studio. I’d rather have everybody with their laptops closed being a great audience.

I’m seeing no open questions. I’m seeing this right at 10 o’clock. Let me close with–

and then feel free to email me, reach out to me.

And Jared, I will hang after and answer this question. But for everybody if you want to move on, I taught my last class at Stanford on the first day of COVID lockdown.

So I had, had students from January to March of 2020. They all did their final presentations in class on Friday. We were live. Over the weekend, we went into shutdown. And I came in on Monday to the first day of Zoom teaching. My colleagues on the faculty were free.

It’s like oh, my God, how do I do this? Where do I get a camera? Where do I do this? I had an awesome final class.

Came into class, set the students up in breakout rooms, sent them into their breakout rooms to watch the GoReact feedback that I had done over the weekend.

Have a conversation amongst themselves about what they saw. Come back in, synthesize the lessons for everybody. Had my final tips, and sent them on their way.

So it was the fact that I was using GoReact for grading made it much easier for me to switch into a digital environment. So with that, Tola, final words. And then Jared, please stay behind, and I’ll be glad to answer your question.

Absolutely. Thank you so much J.D. For such an enlightening presentation on how you incorporate your feedback to enhance learner outcomes. I’m sure there are just many valuable takeaways for our audience today. And thank you so much to our participants for joining.

We invite you to join us again in about half an hour here at 11:30 Mountain time for our closing keynote by Dr. Mordecai Gumley. Thank you so much.

Thank you all. So Jared, I’m looking at your questions, and it may also be that some of the folks from GoReact may have thoughts on this too. Does GoReact have a collate feature of student comments for an assignment similar to the discussion board or another option I may not currently be aware of?

I can just tell you what I do, Jared.

I assign them up front.

You will be evaluating team 3 and team 5. And then when my TA loaded the videos, I just had the observers listed as presenters. I didn’t want the presenters to see the feedback yet.

And so the students went in and they wrote all their comments. They had like–

I don’t know, 72 hours to get their comments in.

And then I took their names off as presenters. I wrote my feedback. And then we added the names of the students who I wanted to see it. And then made it available to them.

The only global feature that I did was just looking at how many comments people had made.

Other than that, Tola or Mr. Anderson, I forget his first name, Arthur? Is he still on?

Yeah.

Yeah, Austin’s still here. I’m not sure that I fully understood the question correctly, Jared. But there are several options for establishing groups within GoReact, and we would be happy to talk more about those with you just to make sure that we’re understanding correctly. Our support team is wonderful if you want to reach out to us. It’s goreact.com/instructor. We’d be happy to just work with you and find the best way to set up your assignments to meet your needs.

Awesome. All right, anybody else in the room with a question? Or I will go take care of my kids.

We have COVID exposure in our house, so none of my kids are at school this week.

All right, thanks, everybody. Bye.