STEM-Inspired Authentic Assessments

Presenter:

Raeghan Graessle is an Associate Professor of Physics at Harper College where she collaborates with her students to explore the wonders of science and mathematics. She is actively involved in the American Association of Physics Teachers (AAPT), supporting educators from kindergarten to university level. As the former president of the Chicago Section of the AAPT, Raeghan is dedicated to enhancing physics education through creative, more equitable teaching methods that recognize students as souls with diverse interests, goals, and aspirations.

Raeghan collaborates with national Two-Year-College teams to create scientific reasoning labs and innovative assessments. Prior to her role at Harper College, she taught high school physics and chemistry for 11 years, earning certification from the National Board for Professional Teaching Standards (NBCT).

Raeghan holds a Master of Education in Curriculum & Instruction from Loyola University Chicago, a Master of Science in Physics from the University of Illinois at Urbana-Champaign, and a Bachelor of Science in Physics from the Massachusetts Institute of Technology.

Transcript:

Raeghan Graessle:

All right. Thank you, everyone, for joining. So I hope I have successfully shared my screen. Sorry about the tech glitch. My humanity is showing. We’re going to be talking about AI today, so I’m all about the humanity part, and the technology, I was set up and then it got me good right at the last minute. I really appreciate your patience today. So I’m here to talk about STEM-inspired authentic assessments, and I just want to thank GoReact for inviting me to discuss something that’s near and dear to my heart.

My talk is a result of conversations that I’ve had with so many people, colleagues, reading articles, and this is most especially a talk that has come from my students. And today, we are going to discuss what is an authentic assessment? The definition of this does change yearly, so we’ll discuss my own definition and how that’s grown. What is specifications grading? Because this is a critical aspect of authentic assessments. We’re going to talk about AI because that has changed how I do a lot of things, including assessments. And as always, we’re going to be centered on the humanity of our students.

So the first thing I would love to hear from everyone is the answer to this question of what do you really want your students to be able to do once they leave your classroom? Kind of imagine it’s three weeks past, summer break is started. What do you want your students to be doing as a result of your class? Can you share that in the chat really quick? And I put some examples on the screen.

I’m realizing that I can’t see the chat. I want to pause and thank Amber, by the way, for being here. I really appreciate you.

You might notice though, and I hope we’re all sharing in the chat and I’m going to look at it later, that a lot of the objectives and goals that we hold for our classes are not exactly, “Students will be able to list Newton’s three laws.” Instead, they’re coming more from a place of being able to apply our information and use skills that we give them in whatever field they might be going into. And keeping that in mind is where we want to start when we’re thinking about assessing our students at the end. So we want to have this strong beginning of knowing what is our real goal and that’s what we’re going to use to build towards an authentic assessment.

So one thing I have a goal for this talk is that you will feel inspired and prepared to create or to change or keep going with any authentic assessments that you are also doing in your class. And the steps to this are just to ask these critical questions. What are your goals? What’s the best way to assess that goal? How should you grade it? Because too often people are telling us how to assess our classes and they’re not helping us with that critical part of grading them. And then what about AI?

My assessments and my definition of authentic assessments are centered… I teach a physics class and most of my students are not physics majors, not science majors, they are not math majors. They are unhappy to be in physics. And so my goal for them is to be able to apply the ideas and strategies that they’ve learned, their critical thinking and problem-solving skills, in their everyday experience once they leave my class.

So I am lucky enough to have in my classroom a whole bunch of future physical therapists and pharmacists and radiologists and sonographers, most people going into health careers, and I really want them to be able to use physics in their career and I know it won’t be directly using our equation sheet, but the thought process is what I want to move forward and that’s what I assess. And so my assessments really focus on doing a case study. I create case studies that are physics-based. One example is having a can of soup roll down a ramp at your home. I teach online physics, so my case studies need to really be everyday life. So if you can roll a can down a ramp, what do you do with that?

And so I teach my students first how to make measurements. That’s the beginning of a good physics activity. How do you make measurements with a ruler, with a protractor? How do you find mass by reading labels on cans or by using a kitchen scale? And from those measurements, my students then answer questions. For my conceptual class, I like to give them question prompts, but for my more advanced classes, I tell them to solve for everything they can. Now, I give them a list of things that they supposedly can solve for. They’re the list of things that we’ve learned so far in the course just to give them some inspiration, but then they go for it. They tell me what the answer is, what can they do?

Once they’ve done this and solved for everything they can, then they create their entry. So how are they going to present this information to me? Are we going to use video to present that information? That’s how I got into GoReact. Or I have other choices that I call U-Pick. Students can choose to present their findings on PowerPoints or on Notability or on a piece of paper that they hand me. What is their talent for communicating? I want to start with that and use that and then they use their favorite talent to show me what they can do. So for some students, their talent is writing pencil to paper, and other ones produce the most amazing PowerPoints and Prezis that I have ever done seen. And it’s so exciting actually to get that variety in your assessments.

The last section is very important and that is to improve. Assessments should be learning opportunities, and so all of my assessments allow students to try again, and I write comments and then the students can read those comments, adjust their thinking if they need to, adjust their presentation. That’s mostly what happens. Their communication was a little glitchy. Then they can grow and learn and do better. So that is my definition of what an authentic assessment is is it’s using something real to your field that I call a case study, then the students create their entry and they’re allowed to improve.

Side note, I was just in that panel presentation, and I love the idea of using ChatGPT to help you set up your own case studies. I just found out that ChatGPT is wicked at those. They’ll be holey, but it’s a good start. So if you’re ever like, “I don’t know how to write a case study,” let’s use our friend AI.

I have recently, so since the last three years, found video to be my favorite way for students to present their information to me because video allows you to judge skills that are beyond the written word. And in my field, physics, all of my assessments require writing, math, drawings, and sometimes actual videos of what did you assess? I want to see that can rolling down the ramp. And so with video, you can judge skills beyond writing. You can see there’s speaking skills, which is frequently something I’m asked to judge on recommendation letters, and you can also use your instincts. We all have them. You can watch a student video and you know if they need another chance at that subject. If they’re reading from their notes very carefully or if all of a sudden, their language got very fancy and they’re reading you a five-paragraph essay using really large words and ideas that you haven’t taught yet, you can, on video, catch that quickly and then allow them to, “Let’s go over that again. Let’s try that one more time. It sounds like you need another chance.”

Videos also allow for students to reflect on their work better because they can see and kind of feel what they’re going through as they answer the questions. And video’s really easy. It’s really easy to use. We have the Blackboard LMS at Harper and video really embeds in that sweetly. And then I have to have a little love note to the 1.5x button when you’re grading the student videos.

Here is a sample entry. This is Quinn’s entry for one of my authentic assessments for conceptual physics. And in that class, one of the topics we cover is circuits, and circuits is not something that I can actually have the students build at home. Believe me, I’ve tried. It was a complete disaster. So instead, we use what’s called a PhET simulation where students build circuits in the PhET environment. PhET is out of the University of Colorado and they present and provide simulations for physics, math, and other sciences that are amazing, and they embed directly into the LMS for free. So my student does not have to click anywhere else. They can literally click and drag wires, batteries, and bulbs right in Blackboard to create circuits. And combined with the video, they can do this assessment.

So what you’re seeing here is Quinn was asked to build a parallel circuit with three bulbs and one battery on screen live and explain what they were doing as they built the circuit. And then once the circuit was built, then Quinn, the next question was to remove a wire and tell me what happens and why. So Quinn removed a wire and then explained with their knowledge of parallel circuits and how they work that the other light bulbs stayed lit and why, and they explained why. And then I have them add a fourth bulb to the circuit and tell me what happens to the currents and the bulbs in the circuit and why.

And so all of those things are happening on the video and so they’re seeing what happens and then they’re also explaining what happens. So I’m getting how well they are at the skill of building circuits as well as their understanding of the physics happening behind it all at once in a five-minute video. It’s very, very cool.

I have added recently a third thing to this assignment where I actually have my students ask AI how and what should happen with this circuit and evaluate the answers. And so I’m going to have more detail on that later, but using AI has really amped up my assessment game.

I am very much into making assessments a learning experience. As a former student myself getting several lower grades on physics assessments, I would usually… Those assessments would end up in the nearest trash or recycling bin after I got them back because I didn’t want to remember the feelings that I was going through with some of those scores. So those assessments for me were a learning experience of humility. I want my students to have their assessments be a learning experience of empowerment. I want my students to be able to redo their assessment.

So let’s say if Quinn added a fourth light bulb to the parallel circuit and told me, “Oh, the bulbs all got dim,” that’s actually not what happens, then Quinn can have an extra chance to try that question specifically again. And I will point Quinn back to some of our learning materials and possibly add more for them to take a look at in that week that they have to produce a new draft and then re-upload. It’s a wonderful thing and it turns assessments into learning experiences.

And for my conceptual physics, I allow them four drafts per assessment. And in my algebra and calculus-based physics class, they get a draft a week per assessment. They have fewer and larger assessments in that class. And most of my students end up getting, in the harder class, three drafts and then they’ve got it, and in the conceptual class, usually one to two drafts and they’re in. So it really does allow students to try again. And I have a colleague from Iowa who deployed this in his physics classes and he told me that his student, one student commented that it’s the best feeling in the world when you finally earn the certification and pass your assessments.

So a note, I’ve been doing a lot of research lately and the educational research does support this approach where you start off with your strong goals for your course to base your assessments on and then using drafts to make it a learning opportunity, and then finally, this thing called specifications grading that I’m going to get into in just an extra minute because I want to talk about some unexpected benefits to authentic assessments.

So before, I used to give your traditional multiple choice with problem-solving section assessment and I gave that to my online students as well. And when I switched to these authentic assessments where they’re doing case studies and presenting them to me with PowerPoints et cetera, I could not believe these unexpected benefits. First, when I have to assess how well the course went overall, I can just look at how many students passed Newton’s laws assessment this time instead of before, it was a test where students could get a 75% by just floating around with some ideas but never really showing off any physics. Now, if the student passes this case study, I know that they can do this in the real world.

My faulty lessons pop. So before I used to notice that, “Wow, 72 people miss number 12,” I would just throw out number 12, say, “Number 12 must’ve been written poorly.” Never would it be, “Professor Graessle has taught that poorly.” But now with these authentic assessments, if my students are all messing up Newton’s Third Law, for example, any science people out there know that that’s a doozy, I need to improve my Newton’s Third Law module. And so I love this. I know what’s going well and what’s not going well based on authentic assessments in a way I never knew before.

I also have a lot more confidence in my students’ ability to do physics outside my classroom because tests were really testing how good they were at taking tests and I want my students to be good at real-world physics, and now I’m confident that they are or are not if they didn’t get it. It really meets students where they are. It uses their talents that they come in the class with. Again, by allowing them a choice in how to do the U-Pick assignments and also allowing them to just bring themselves to the test with video is huge and it works for all my modalities, including HyFlex where I mix in-person with online learning in one crazy classroom.

Honestly, y’all, I cannot give another old-fashioned test. I am going forward with this authentic assessments.

So how would you go about… Because we have so many talented people in so many different fields on this call. For making it your own, really you want to start with that goal, the real goal. And also though, I mean, we all have goals too, like students should be able to use Newton’s laws is one of mine. And so if you have that mix of goals, that’s where you start. And then you can create a kind of rubric of what are the real things you want your students to show you they can do? No partial credit. So what is specifications grading? What is this no partial credit? That’s a key part. And then developing that case study. Again, I am tomorrow going to log into AI and see how that can go. But developing those case studies that you have, that’s the real seed of goodness with these assessments.

Okay, so specifications grading is sometimes called specs grading and it has really changed my life because authentic assessments, the biggest argument against them is the time sink. Let’s talk about how to balance the grading for these things.

So specifications grading is really based on the idea that you don’t want to be mired down in giving partial credit for things. I bet you can tell at a glance, even if it’s a long problem to solve, if an assignment meets or doesn’t meet expectations. You can go granular too. I have lab reports and my friend Chris Louie introduced me to this idea where, okay, lab reports require a hypothesis, for example, and my hypothesis was worth three points and then I was agonizing over if someone just didn’t get the hypothesis, should they get a 1, a 2, which one? And then you know the students are going to be like, “Why did I get a 2 instead of a 2 on this?” And all of those things going into it. So let’s just stop. They either did a good hypothesis or not yet, a 1 or a 0, and that then you read the hypothesis and you’re like, “1,” or you’re like, “0.” Very fast. No more agonizing over that.

So also, I mean, I have students who are famous for doing a physics exam, getting a C, even though they didn’t really do any physics on there, it’s just my rubric and my partial credit, just all this. Oh my goodness. It was just very hard to stand behind. But with this, it’s much faster and easier.

Here are some of my rubrics where I look at they either did it or they did not in a physics context. So for example, in forces, I want my students to be able to do calculations, and then I also want them to show off some skills. So I have different layers of assessment there. And calculations would be stuff you calculate, like in your calculator or explain to me why you can’t do that. So I want them to do seven calculations and show all these skills. And so I tell the students, “Solve for everything you can about your case study,” and then I give them these, what I call, rubrics that are really just the list of things I want them to be able to do.

So for forces, the list is pretty big. There’s 11 items that I want them to be able to do. It’s a six-week unit, so that’s pretty okay. For motion, it’s a little shorter. But these are the rubrics that I give the students so that they know what I’m looking for specifically when they solve for everything they can. And this is for the more advanced courses. For my conceptual class, the rubric is just the list of questions that I ask them to answer, usually on video.

Bundling assignments is a part of specifications grading. So when you’re grading a single assignment, you can give every little bit a 1 or a 0, but then how do you apply this to maybe a whole chapter of your class, or even your whole class itself? So I have a whole bunch of examples of bundling in my class.

For example, my first thing here is what if I just say to the class, “You know what? If you pass forces, motion, and energy momentum unit, you get a D in physics. Then to get a C in physics, you have to pass rotation also. To get a B, pass fluids also. And to get an A, how about all the topics, you pass?” Or if you want to focus on percentages, you could say, “A D in physics is you passed the lab section, you get a 33% or higher on homework, and 50% or higher on quizzes. That’ll give you a D.” Or if you want to just bundle a smaller section of the class, you can say, “I’m going to bundle the lab section of my class and say you get a D in lab if you pass eight of the labs.”

You can also get a little complex if you like your and/or computer science technology and use, “You can pass lab or the quiz section. You can complete 50% of the homework.” You can bundle away, but bundling organizes your class to make grading faster and streamlined and it’s wonderful for that work-life balance.

A really big part, again, I’m back on this redemption and flexibility thing is just allowing students to try again. If you’re grading with a 1 or a 0, some students, it’s a little bit harsh on them. I call it, “You got it,” or, “you didn’t get it yet.” And I like to… This virtual token economy works for so many people. It does not work for me because it’s hard to organize, but if you are an organizational maven, check this out. Students can use tokens to revise any assignment that they might’ve got a 0 on and try again, as many or as few tokens as you wish. I just have a warning on there of this can get wild. Go at your own risk and find your own level with tokens, but students adore being able to redo something for a higher grade.

My own class, I bundle what’s called the assessments, so I don’t give any exams or a final exam. Instead, I have my students do these authentic assessments throughout the course, and I have students vote on what percent of their grade the assessments are going to be. And on average, they pick 35% of the grade to be the assessments. And then this is how it’s scored. I use the bundling. I figured out what are the most important units of my course?

And the most important units in my Physics I class are forces, motion, and energy because they’re going to use those ideas everywhere. My other units, I’m going to be honest, rotation, huge for my physical therapists, not so big for everyone else, so rotation’s the next most important. And then waves, huge for my sonographers and radiologists, not so big for everyone else, so waves is kind of tied with rotation in importance. And then fluids, sorry, fluids, you’re really cool, but… So that’s how I have this bundled. So forces, motion, and energy, key things. To get a passing grade in my class, you have to pass those three at least.

Then you can add on whatever topic you happen to be the most talented at. For my class, rotation is actually really hard. A lot of students would prefer to earn their grade showing off their knowledge of waves or fluids. And so this gives them that opportunity where maybe if rotation was like, “Whew!” for them, but they’ve got talents in all these other physics fields, they can still get even a B on this section of their grade. Finally, to get the A+, I have a synthesis entry that has them show off how they use multiple course ideas across one situation. So for example, using forces and energy to figure out how fast that can is rolling when it gets to the bottom of the ramp.

I really recommend trying specs grading to everybody because it’s so manageable. You can try it on just one assignment or you can go nuts and base a whole class on it. I’m in the middle. I use it for labs and for my assessments, but I have other areas of the course that don’t use the specifications grading. But the main thing is that it allows you the freedom of doing authentic assessments without that huge anvil of the grading on top of your shoulders, and I think it’s a really cool thing to check out. I do have a good resource on where to start with those at the end also.

We got to talk about AI because that’s a thing. And I’m going to tell you, I put my old test and current homework questions into ChatGPT and I asked it to show its work and it’s good. It’s scary good. It shows math work properly. It’s not quite at the level where I would give it a higher grade than a B because it’s not using diagrams properly yet, but it’s going to very soon. It does make common mistakes because it’s using the whole world’s information to base its information on. So if the world doesn’t understand Newton’s Third Law, guess what? Neither does the Chat. So it does make common mistakes, but that’s where things get interesting. And finally, AI is really charming. I have named mine and I just had to add that because that surprised me how I all of a sudden became friends with a robot.

I really strongly believe that we need to show students how to use AI and how to write proper prompts in our classes starting tomorrow, yesterday, today. This is going to be a part of their life. And if we’re teaching them how to use it, then we are part of their education because that’s our job. So I have a few basic rules that I teach my students on how to write a good prompt into a chatbot and I have them here: be specific, try the phrase “act as if,” use “do” and “don’t,” and specify your output.

So I have an example here of a prompt that we could use on that circuits example from before, which is, “Act as if you were a college physics student and answer the question, when adding a fourth bulb to a series circuit that already contains three, what happens to the circuit and why?” So I’ve given it the question and who it’s answering on behalf of. And then I say, “Please keep your answer to less than eight sentences.” It’ll go on and on if you let it. And then, “Do include the words current, voltage, and resistance.” So this is helping them learn how to get a reasonable answer out of the bot.

Okay, so we taught them how to use it. What is going on? Well, I’m going to tell you AI is not a human. So when it comes up with an answer to a question, we can really rip it to shreds without worrying about being polite, without worrying about social norms. We can just be like, “You dumdum, what were you thinking?” And students, they let it have it. It’s wonderful. They don’t have… It comes without that hesitation to answer a question. You look at what AI says and you’re fine with being like, “What a dunce, et cetera, et cetera.” I love that because then students are more comfortable with critiquing it.

Second, they critique it. I did not have evaluation-level questions on my assessments before AI. And now, I have my students type that prompt into AI at the end of their circuit assessment and then read the answer on screen on video. And was the answer correct? How do you know? So I want them to think about where they’re getting their information from. “I know the answer is correct because it agrees with the textbook” is a starting point and I want them to think about that. What parts of the answer made sense? What parts did not? So I want them to critique AI’s communication skills because AI will dodge a question. Oh, it’s good at that. And then how would you rewrite the answer to be better? I want the students to rewrite the answer.

And honestly, AI overall is strengthening my course. So instead of being this scary thing that I was really nervous about, it’s become a tool that I’ve been using in my course to make it even better.

And then another aside about AI is I have class discussions often and AI allows, if I ask AI to answer the question first before my students do, especially in discussion boards, it allows students to bounce off of someone else’s answer. So if you ever have the student who’s like, “No one’s answered the question yet. I’m too scared to post,” guess what? I got the AI. AI’s answered the question. And so it helps students really figure out what the question’s really asking. What if you have a student who’s an English language learner and isn’t sure how to answer that question? They see a sample answer and that’ll give them some more confidence. I have confidence also there for people who just have low confidence, seeing someone go first is a big help. And finally, if you have to go out on a limb with a difficult topic and you are scared of answering it wrong or offending people, this is a wonderful tool.

So I have AI answer this question inside of my social justice and physics unit first before the students talk about it. Why is racial diversity important in the field of physics? You can see that question. You can understand why you’d get crickets if you ask that to your students, especially my students tend to skew white and male, and this, having AI answer the question first, this is where AI actually dodges the question. Then the students get really uproarious about it, “It dodged the question. Why did it dodge the question? What do you think it would’ve said if it didn’t dodge?” And then all of a sudden now, we’re talking, we’re talking about that tough topic. It gives us a way to do it.

So I really would love for you to take a moment and if you have a favorite way to use AI in your class or in your position at school, if you could share that in the chat, I am going to take a look. I would love for you to do that here. And while you’re sharing, because this is the end of my talk, I just want to say that authentic assessments aren’t just authentic to your subjects, but they make space for the authenticity of your students too. I had a student tell me that in my class, she no longer felt like a robot. She felt like a human. And that was the biggest compliment I think I’ve ever gotten.

So I just want to say thank you to these wonderful people who’ve helped me with my ideas and talk today, and also show you two articles if you want to know some places to get started with AI and also with specs grading.

So I think we’ve left a little time for questions, right?

Matthew Short:

Absolutely, Raeghan. Thank you so much for the presentation. And we do have one question in the Q&A from, I believe, a friend of yours, Melissa, “What are your favorite ways to give feedback on student videos?” She does include a small plug for GoReact and the feedback features there, but anything on top of GoReact that you’re using to provide your students with feedback?

Raeghan Graessle:

My students always upload a physical visual aid and their video. And so I always do a combination of… I like to give a video back to their video just as a, “Thanks for your video. Here’s one from me too.” And then on the visual aids, I love doing screenshots of specific parts of their visual aid to tell them, “This part was really excellent,” and then I’ll use another screenshot to say, “Uh-oh, you missed a negative sign here,” and things like that. So those are my favorites.

Matthew Short:

Wonderful, wonderful. And I do see a comment in the chat from Kristen. “The best piece of advice, other than what I just heard, came from a Microsoft AI conference. Ask the AI how it can help with your job. You’ll be surprised by its answers.” I like that. And I also really liked your idea of letting AI kind of be the one that steps out on the ledge to answer the question first. I love that suggestion and feel like that’s a great way. And to your point, you can pick on AI. Its feelings don’t get hurt. It doesn’t have to be brave. So letting it lead the way and letting students respond, I really love that suggestion.

Raeghan Graessle:

Thank you.

Matthew Short:

Any other questions from the group? We do have maybe one minute if there are any last questions or anybody else wants to share how they’ve incorporated AI or authentic assessment into their coursework. So please don’t be shy, add those into the chat. We’ll give just a minute to see if there are any last chats or questions.

Let’s see. And I see from Sierra, “I gamified AI assignments.” Gotcha. Very nice.

All right. And I think that might be it. Oh. Oh, there’s from Melissa, “Really fun to tackle a problem or a question in creative way. For example, answer in the form of a poem or a parody.” And now I’m just thinking of “Weird Al” Yankovic at this point.

Raeghan Graessle:

I know! That’s where I went too.

Matthew Short:

Love it. Love it. All right, so with less than a minute here remaining, I do want to thank Raeghan so much for her presentation today. This has been really helpful discussing authentic assessment and the different way that you can incorporate AI into your coursework. Really loved this. Thank you so much for sharing this presentation with us.