Redesigning Assignments by Incorporating AI Tools

Derek Bruff:

One of the assignments was for a writing seminar that I used to teach, a course on cryptography, a course about codes and ciphers. It has a mathematical component but also kind of a history component, certainly some writing and also current events. It’s a really fun course to teach. And so I had an expository essay that asked students to find a code or cipher from history that we hadn’t already explored in the course, and to tell its story in 1,000 to 1,500 words. Where did it come from? How was it used? What impact did it have? But also how did it work mechanically?

So there were two main goals for this assignment, one was a storytelling, how can you tell a compelling story about this little slice of history, but then also a kind of technical communication piece. I wanted to see if students could explain the mathematics behind some of these codes or ciphers in writing. And a little bit of research as well here. This wasn’t a full on research paper, but they did have to find some credible sources. And so I went to ChatGPT and I said, “Well, what if I just ask it to write this paper for me?” One of the ironies is that I try to be really transparent with my students about the assignments and what I’m expecting from them. And so because of that, it was pretty easy to copy and paste my assignment instructions into ChatGPT, and it gave me a response here, I screenshotted this.

It was not good, this was not a good essay. It was very by the numbers. It picked a topic that was actually not a good topic. The Enigma machine is a fantastic topic in general, but we had already talked about it in our course. And so it was not fair game for this paper assignment. But more importantly, this essay was really dull and very formulaic. I used my grading rubric on ChatGPT’s essay, and it scored a D minus according to my grading rubric. So not great. But I thought, what if I were a student who either really wanted to get out of this assignment or maybe just wanted to use these tools effectively? How would I keep using generative AI to try to help me on this assignment? What else could I do? Well, the next step was the easiest, I needed a different paper topic because the Enigma machine was not appropriate. And so I asked ChatGPT to identify some other codes or ciphers from history, and it did a pretty good job of generating [inaudible 00:14:29] many of which would’ve been appropriate for this essay assignment. So that was good to know.

I picked one of them, the Japanese purple cipher, which was fair game for this assignment. And I went to a different tool called Elicit. Elicit, it’s going through a lot of changes right now, but as of a month ago, I was able to put in a question like what is known about the origin, use, influence and mechanics of the Japanese purple cipher? And what it does is it actually looks for scholarly sources that might answer that question. And again, these tools are good with words. So they’re able to take my question, which is expressed in words, and to try to find papers or book chapters that might answer that question. It gives me citation information about these resources, but then also in case I get tired of reading those long abstracts, it will summarize the abstract for me, which I think is kind of comical, but it will also do other things to try to pull out different features of these papers.

So I view it as kind of an alternative to a Google Scholar search or a library database search in terms of finding resources. And I looked at this and I said, “Okay, some of these are pretty good.” Now I’ve been in this field for a while. I know where to find good sources. My students do not. So my students would need to do some digging into these suggested sources, but I saw a few here that had some potential that my enterprising student might want to use. So then I went to Bing Chat, which is another generative AI tool attached to the search engine no one used before last year, Bing. And I asked it the same question, what do we know about this cipher? And it gave me a nice summary of different sources. And it’s important to know that Bing as a search engine with a chatbot will search the internet and will find live sources out on the internet and will summarize them for you.

Again, the summary is actually pretty good generally, they’re good with words. And it gave me some sources that I could follow. These sources were not as strong, it’s just searching the general web, it’s not searching scholarly sources. And so several of these sources would need to be interrogated because of that. It also though, as you’ll see at the bottom, it gave me other things I might ask it about this topic, which I thought was an interesting exploration tool. So then I went back to ChatGPT and I said, “Let’s take another approach. Instead of just giving it the assignment description, I will ask it to tell me an exciting story about the breaking of the Japanese purple cipher.” It did much better at this. This was a much better essay. It lacked the kind of technical communication that it needed to have, but the storytelling was very strong.

And so where I landed was that a student who wanted to use these tools thoughtfully could kind of mix and match output from different tools and probably write a pretty decent paper. So if it brought some of the technical communication from the first draft, the storytelling from the second draft, and then built in some references and use of those sources thoughtfully, a student could use this in their final paper and probably do a pretty good job. So I asked myself, what are my makeover decisions here? What would I have to change about this assignment given all of this? One is that I would decide to teach writing with AI. So I’m not going to try to avoid the use of these tools. I think they’re going to be everywhere. They’re coming to Google Docs and Microsoft Word, and so I need to help students learn to use them well.

There may be some courses out there where you’re going to prohibit the use of these tools. I talked with a philosophy professor recently who was doing a graduate level course and her students were basically doing almost publishable level work. They weren’t using AI. But she had a small class, advanced students who knew why they were in there, and she was able to tell them specifically, “This is why we’re not using AI, because I need you to develop these skills for your profession.” But for my first year writing seminar, I’m going to use the tools. As a class activity, we’re going to collaboratively evaluate that first ChatGPT essay draft and use it as a way to talk about what’s not good about this kind of writing. We’re going to spend some class time on structured peer review. So I have some activities where students are going to give each other feedback about their storytelling and technical communication choices. So even if the tools are doing some ghostwriting for the students, they’re still going to have to have some discussions about what rhetorical choices to make in their writing.

We’re going to use those source suggestions to teach about working with sources. This is something I would do anyway, but Bing Chat just gave me some terrible sources to evaluate, and so I want to capitalize on that and use that as a way to help students judge the quality and credibility of sources. I’m going to ask students to document and reflect on their use of AI. So I often have students do a kind of writers statement or designers statement alongside a project, and so I’m going to want them to kind of talk to me about AI. I want to keep those lines of communication open with my students. And I’m going to look at that rubric and see if there are any categories I don’t need to spend points on anymore. There may be some stuff around grammar or fluent prose that just isn’t worth giving points to anymore because I can assume that students will use an AI tool to clean up their language like that.