Authentic Assessments for the AI Era

Jenny Gordon:

We’re thrilled that you’ve joined us and we hope that you enjoy the presentation and walk away inspired to reframe your assessments and to drive meaningful and relevant learning and skill development. I’m Jenny Gordon. I’m the Vice President of International Markets here at GoReact, and I’m coming to you live from a very, very wet, rainy London on this afternoon. But I am very excited to begin today’s prerecorded discussion. Our fantastic guests, Bert Verhoeven and Vishal Rana are based out of Flinders University in Australia, and I met them last week to talk about authentic assessment. I’ll be answering any questions you might have about this topic during the session today, either whilst the panelists are discussing, whilst you are watching recording, or at the end for you. For those of you who aren’t familiar with GoReact, we’re a learning technology solution. We’re very, very focused on improving learning outcomes through the use of video and feedback within the assessment process. We’re currently used by more than 800 schools, colleges, and universities and professional organizations from around the world.

Today’s discussion focuses on the growing disruption that AI is causing in higher education, about embracing the possibilities of AI, not just the worries that it might bring, and the promotion of authentic assessment, experiential learning, and the use of technology. And I ask, could AI be the catalyst for an education revolution that provides efficiencies and improved outcomes while preparing us all for the future? So let’s get started with a poll question. Are you reevaluating your assessment strategy in light of AI tools? You can give us a quick answer to that. The poll question is placed there. Are you reevaluating your assessment strategy in light of AI tools? We will have a little look at those poll results in just a second once everybody has given us their answer. And thank you very much for taking part in that poll. Now let’s meet our stars for today. Bert Verhoeven, sorry, and Vishal Rana, great panelists. Let’s listen to our prerecorded discussion that took place just a couple of days ago.

So hello and welcome, Vishal and Bert, and thank you very much for joining me today and for joining the GoReact team. Before we start our discussion, which will be centered around authentic assessment in this ever and fast evolving era of AI, I’d like you to introduce yourselves to our global audience, but can I ask you to introduce yourself first, please?

Bert Verhoeven:

My name is Bert Verhoeven. I’m from Flinders University in Adelaide, Australia, and it’s winter here now outside. It’s light, it’s beautiful. And I’m excited to be here and thank you, Jenny, for asking us to come and talk at your webinar.

Jenny Gordon:

Thank you, welcome. Vishal.

Vishal Rana:

And my name is Vishal and I am based on the sunny Gold Coast in Australia. We also have winters, but as you can see in the background, it’s very nice, bright and sunny. And I’m working at Griffith University as a lecturer in management in the Department of Business Strategy and Innovation. And once again, I concur with Bert. Thanks, Jenny, for inviting us at the GoReact webinar. Thank you.

Jenny Gordon:

Thank you very much for joining us. We’re going to have just a great discussion today around all things authentic assessment and the evolution of AI. Certainly we’ve just had a great introduction conversation around how quickly things are moving and how we hope to see a real revolution within the education space. I know that lots of the ideas, thoughts, and opinions that we’re having today are very much based on your own, even though you are coming from different institutions today. We’ll be also sharing out some links to some of the articles and blogs and publications that both Vishal and Bert have published ahead of this webinar.

And while we’re speaking actually, there’s been more published as well, so we’ll be sure to put those links out with this information as well. So let’s get started. I don’t think there’s a clear answer to this one, but let’s start it in terms of a discussion. Vishal, what do you think are the biggest disruptors, and some may say threats, and you do call that out in your recent paper, what do you think are the biggest disruptors that AI is having to the higher ed space at the moment?

Vishal Rana:

Great. Before I answer that, I just want to tell your audience that everything that we say here today is our own opinion. It’s got nothing to do with our university. And with that, the biggest disruptors as we see is technology, the AI as we call it, and the way it’s been implemented in the university or in the education system altogether. The beginning of generative AI, we are seeing how powerful of a language model that is and how it is disrupting the assessments. For example in the universities, the way we teach in the universities, the way we go about doing pretty much everything in business and it’s quite powerful in that way.

Jenny Gordon:

Bert, do you have anything to add?

Bert Verhoeven:

Yeah, I think the disruption moment will be in the apocalypse of cheating that is coming over us in the second half of this year. That will be the moment where people start realizing that we can’t do this anymore. But of course it’s been going on for much longer than that, and AI goes much deeper than just assessment. And so I think for me it’s very important to see that there are certain elements that are very disruptive like, for example, assessment and students using it to cheat on exams or surveys or of essays or reports. But I think it’s a much broader discussion and a much deeper discussion about how we teach our students and how we prepare our students for their roles as knowledge workers in the future. And I think that is a much more interesting discussion than the cheating discussion, which is going to happen. There’s no doubt that is already happening and then will come. The apocalypse will come and that will lead to change. So, this is good.

Jenny Gordon:

So on that note then, what should institutions be doing to ensure that students are learning and not just… Or learning with AI effectively? What can we be doing to do just that, to ensure that it’s more than just approaching the cheating discussion, but looking at what AI can do to really take us to the next level in terms of teaching and learning?

Vishal Rana:

I think we started doing that in our courses in the beginning of the year already when we, Bert and I, because we work together on the courses in innovation and design thinking, we already started embedding the way we go about using AI as a tool rather than just looking at as a cheating tool per se. Because again, it depends on the kind of assessments that you are providing to the students. If it’s a normal essay or anything like that, a report, students can easily put in some keywords, the prompts as we call it, and they can have the answers generated. Although, they started with an earlier version where they could have fake answers to say, but now it’s getting pretty good, it’s pretty accurate. The fakeness is almost gone. The hallucinations, as we call it, are almost gone and the models are getting better day by day.

So we actually use that as applied learning, so as a tools to be applying while learning. And then for example, we use it for design thinking and solving a world’s greatest problems. And how could actually they use it as a tool to create an empathy research, for example. So they have to come up with empathy questions in our assignments and they have to go and interview people to understand the user problem in the design thinking what they’re trying to solve. So these questions could come up from the empathy interviews from using AI. It just became a copilot, as we say in the learning journey, rather than just focusing on using as a cheating tool. And I’m sure Bert can add more to it.

Bert Verhoeven:

Yeah, I totally agree, Vishal. I think what we have done is we have integrated from the first semester here in Australia, which started in February until now into all our teaching. So we have said to our students, you have to use it, it’s not an option for you anymore. You have to use it in everything you do. You need to use AI. And we gave them a framework for that and we taught them how to use it as well. And we learned from teaching them because it’s so new, we were doing our own experiential learning ourselves and learning by doing. And I think what we learned from that, my main learning from that is that our roles are changing. So our roles as teacher or as marketing manager or as educator, it doesn’t matter what kind of knowledge work you are, our role has changed since the introduction of ChatGPT or other large language models as a learning tool and as a copilot.

That is I think is the core of this discussion that will never be the same again. Our roles will now forever be changed and we will not be creator anymore. We don’t create tweets if you’re a marketing manager, but you will now ask ChatGPT to create 10 tweets about a certain topic, then edit it, then give it back to ChatGPT, ask to improve it, then edit it again, then test it to people, bring it back to ChatGPT, asked to edit it again, come back again, and then facilitate it into the real world. So our role changes from a creator and a manager and an administrator to become a co-creator with AI as a copilot. An editor editing what you create together and facilitate into the context which AI is not very good at.

So we facilitate real life context around us, our authentic environment, which AI cannot do yet, at least say yet, but not for a while I believe. So if we understand that that role is changing for all of us, then we need to learn differently as well. We need to learn how to be a co-creator, editor,, and facilitator. We need to learn how to collaborate, we need to learn how to edit and we need to learn how to facilitate. And that is, I think is the core of the changes that are going to happen this year, next year.

Jenny Gordon:

Absolutely. And we said earlier when we were chatting that, we’re talking about higher education today, but this is in every part of life. It’s in every part of education, it’s in professional, it’s in everything. And there’s inevitable fear when it comes to change as there is with anything, and the time it takes to make those changes varies. But what we’re seeing is this very quick introduction of different tools and services while also maintaining the delivery of courses and functions and things like that, and assessment. Regardless of the introduction of AI, assessments in this year’s higher education calendar will go on. So what are you seeing when it comes to authentic assessment and how to adjust what’s happening today while we’re still wrapping our heads around the broader impact of AI? How can authentic assessment be included or discussed or incorporated into the assessment process now so that students are getting the best and most effective experience as an assessment within higher education?

Vishal Rana:

I think authentic assessment is not a new way of doing things. It’s been there for a while in the academia. It’s just not been widely acknowledged and not everybody wanted to use it because it’s a lot of work. And that’s what we have seen in our innovation course. Experiential learning takes a lot of effort, not only from the students, but also from the people who are delivering the content. And it’s almost three times the effort that a normal course would take in a university or in higher education that we put in for authentic assessment and the learning process, because there’s checkpoints in every stage of the learning when it comes to authentic learning because it’s a feedback loop. You do something, you get feedback, you then fix moving forward or you go back, you revisit. And that is a lot of effort and that is something that I think universities have to start acknowledging and accepting that the effort is not going to be reduced.

It’ll be more effort, but AI will make it more experiential, more better as a copilot in the learning process. And again, as Bert said, we’ll be the co-creators. And you nicely picked up, it’s not only impacting the higher education, even in schools. Recently, if you heard Khan Academy came up with a new AI tutor called Khanmigo. So you could be sitting at home and this AI tutor will be always there with you 24/7 in your own way of learning. Everyone has a different way of learning everyone. Somebody’s far, somebody’s slow. I was very slow learner. I wish Khanmigo was there with me when I was learning, because it actually goes through the process of getting you to understand and lets you think critically. It doesn’t give you the answers like we have been telling, that you just can go and cheat and put in the prompts.

It actually gets you step-by-step process in your learning journey, it suits everybody. If you’re a morning learner, a night learner or a daytime learner, it’s always there with you. It’ll be the same with higher education. The AI tutoring will come into higher education or has to come to higher education where students who want to study at night, they don’t have to wait for us to respond back and the next day or 48 hours after that, that model is gone. They want answer immediately and this AI tutor system will provide them with the answers. So it’ll just enhance the learning process for them. And that is already happening with Khanmigo. And then some universities will adopt it faster than the others.

Bert Verhoeven:

Yeah. No, I agree, Vishal. And then what’s most interesting I think is that this is happening now already. So this is not something that will happen in 10 years term or five years, three years.

Vishal Rana:

No.

Bert Verhoeven:

It’s not something…. We’re not talking about the future here, this is now. And although the interfaces will become better, the personal tutors will become easier to work with and better interfaces and it will be much more instantly and then driven by voice and et cetera. So the interface becomes better. But if you use ChatGPT now as a personal tutor and if you know how to prompt it, you already get the results that you’re after. And that makes it a personal tool and it makes it a tool for personal learning, and that is the big revolution that is coming to us. That means that the result, and we have already noticed that in our classes, the student outcomes are better than last year.

So the results that students come up with, the level of the results, are they doing some research in design thinking and then they look into projects or sustainable development goals and they look for a problem and they find a solution and they present it. And that’s in a nutshell what they do. And the results of that are better, their research is better, there’s more depth in it. There’s a bit deeper thinking required nowadays than in the past because they can use AI as a learning tool and as a tool to help them. That also has the consequence that the level that we require to finish the course will also be at a high level from now onwards.

So there’s pros and cons. It’s also that everybody will have to study more and better and at a higher level. And that is sometimes very difficult. So we need to really rethink that as well and say, well what kind of roles are there that required this really high level thinking skills? And I think co-creating and editing definitely required that. Where the facilitating role may be done by people who may struggle a little bit with those higher thinking skills. And so again, it requires a whole rethink of what we do in our knowledge work. And most of us are knowledge workers nowadays. We’re all out there.

Jenny Gordon:

But it has to be that goal of increasing quality, which is ultimately what you’re summarizing there has got to be that ultimate goal and quality, of course. But the readiness for then career readiness, workforce readiness, and the application of different assessment can then make graduates more employable or at least better fixed when they leave education. Can you-

Vishal Rana:

We are-

Jenny Gordon:

[inaudible 00:18:12] Examples of where you’re seeing that happening already with the use of authentic assessment as well.

Vishal Rana:

Just before that you picked up very nice point there, Jenny, is the organizations will start using people, start advertising jobs that require these graduate attributes where students need to know how to do better prompts. And we have actually in our article already shared some of the jobs that have been advertised in Japan where the employers were asking students, if you don’t understand how to do better prompts, don’t apply.

And this, we’ll start seeing that more often from here on because prompting is not easy in itself. And we learned through our own journey of teaching, the better the prompting kept going, the better the results started coming. So that will be a whole new requirement from the organizations as an employability skill. How good are you at prompting? Because prompting takes a while if you keep playing with it, but those who are better at prompting, who know straight away get into the job and what kind of questions to ask, they’ll be more in demand. And again, this is what we already started training in our courses on how we can do better prompting and teach students how to do better prompting as well. So I hope that answers your question.

Jenny Gordon:

Absolutely. It’s that mastery of any sort of skill, isn’t it? The mastery of a skill that you can then apply. And that happens sometimes in great depth and in other times not so much. And I think that this sort of convergence of AI and knowledge base, it’s enabling what was more traditionally knowledge based courses, knowledge based learning to be considered differently. We talked briefly earlier about in the UK, vocational learning is very big and much supported. We had some government chatter about the importance of apprenticeships and really learning on the job, that sort of thing. And the way that students that go down that vocational route of learning are being assessed as they go very authentically using lots of different tools and services.

And we see that the rise of different vocations as we look at the skills crisis and the jobs that aren’t going to be current in the future or those that we don’t yet know exist. Where do you think we’ll see the biggest change as a result of the combination of AI and authentic assessment? Which sort of vocations or disciplines? I know Bert, you’ve talked a lot about creation and editing and that sort of thing. What about other disciplines or vocations where you can see that there’s going to be dramatic revolution or change happening quite soon?

Bert Verhoeven:

I thank you, Jenny. I spoke at a communication seminar last Friday here in Adelaide and we were having three universities together and people from university talking about this, about communication for example, where communication is very much under threat from AI. AI is very good at making content, writing content, editing, improving things, helping writing marketing plans, executing marketing plans, it’s very good at all that. So we were then having the same sort of discussion that I said here, and this is what I’m saying with their roles, they applied to almost every field. So we are having the same discussion where we say, okay, instead of creating the content yourself, you will have to co-create it with AI from now. To get a better quality, to get a higher quality and get more production going, you will have to start doing that with AI.

And AI will help you increase your production as a marketing manager, increase your quality as well. If you do it well, right, if you go back and forward, you edit it and you go back. So that is a whole process of prompting, not just one, but the whole process of it. And you go back and forward, and then if you do that well, you will be more productive than other marketing managers. You will be more innovative than other marketing managers and your outcome will be better, you’ll have better results, your marketing campaign will work better than the average marketing campaign and that will make you stand out. So the way I see it is that innovators who learn how to use AI and learn their new roles in that and they’re open about it as well, and employees and employers are open about that too, will I think benefit the most from this, because they will learn the most from it while doing the experience learning on the job. And they will also achieve the best results and then be more valuable for clients.

So I think the core is that there… And this applies almost to every field. So it could be education, could be communication, it could be anything out there, healthcare as well, doctors. Our student teams were already designing AI doctors and when they spoke to young people and old people and the old people were sort of a bit hesitant about that. They’d rather talk to the real person where the young people said, oh, if I have to wait two months before I can get an appointment with a doctor here in Australia and I can get it straight from an AI and they can also prescribe me and do the same quality research, then we don’t care if it’s an AI or not. So it’s very interesting, that kind of attitude changes I think will happen super fast. And that is, again, not five or 10 years, that will go much faster than that.

Vishal Rana:

That is a question that is asked, thanks for bringing that example, but of doctors. So the AI doctors now, I was listening to a conversation by Emad Mostaque from Stable Diffusion the other day and the question came, what is the future of the GPs? Why do we go to GPs? And it was an interesting conversation because every day we publish 7,000 medical journal articles every day. How many medical journal articles does your GP read on a day-to-day basis? Even if one can’t compete with the AI that has read 7,000 journal articles every day. And so it’s not that the GPs will go away or maybe eventually they might, but it’ll be, would you like to go to a GP that uses AI as a tool, as a copilot while they are diagnosing? Or would you like to go to a GP who has got a limited amount of knowledge?

And that is the question to ask. And how long will these GPS be required? Because again, it’ll be the governments who will try to stop these technologies to infiltrate into getting the GPs away. But eventually in five years time, you might not want to go to GP because your AI would be able to diagnose you better than any human doctor has ever done. And you already started seeing these examples for the last 10 years in image recognition and all of that, that AI is better able to diagnose potential cancer than the human doctors with 99% accuracy. So that’s where we are heading to. Not only that, lawyers. We had some law students in our class last semester, quite a few of them. And we had an issue with law students where I had an issue, I built a house recently and I was having an issue with a contractor where my retaining wall subsided a little bit and I was trying to get the contractor to come for a month, “Can you come and help me?”

And he just said, “Yeah, I’ll come, I’ll come, I’ll come.” But it’s been a month and I’ve got other things to do, so I went to my law students, I said, “What should I do?” And they said, “Oh, you should just go to legal aid or consult a lawyer and see what can be done.” I said, “Let’s do an experiment.” In the class I just opened up the generative AI, the ChatGPT, and I put up the whole scenario in the ChatGPT and said, can you create a legal document for me enough to push the contractor to think that this came from a lawyer and it’s taking to the next level. I did that, I sent an email. Two days later, the contract was at the house. He thought that it came from a lawyer. I didn’t have to go to the lawyer and things got done straight away.

And so then they asked the question, “We are going through this four years or three years of law degree, what is our future when AI could do this sophisticated…” And this is still early days, I’m talking about March, April, may, and now we are in July and it’s already become more sophisticated in the way it responds. And so there’s a lot of professions, especially anything to do with lawyer work. Anything to do with law.

Bert Verhoeven

Customer service as well. There’s numerous examples now where the general opinion seems to be that AI has no empathy, it can never be like a human, never have the same warmth. Well, so much research is showing now that it’s not true. There was this green energy company in the UK that used AI to answer their emails of their customers and the people who are their clients and they found that the response from AI, so from ChatGPT had, how do you say that?

Vishal Rana:

The customer satisfaction rate went up.

Bert Verhoeven:

Yes, customer satisfaction. Thank you, Vishal. The customer satisfaction rate for a human was 65%. The customer satisfaction rate for AI was 85%. So this was after thousands and thousands of emails. And the same with doctors, so the AI doctor, people thought they had more empathy and they were more empathic than the real doctors. So I think where we think that humans have an advantage, which I think in practice doesn’t appear to be through. And so we really need to think about what we do well as humans and where we have an advantage and what our role then is. And I think in many ways we can learn from AI in that sense. And certainly in terms of education or in terms of empathy, we can learn a lot from AI I think, in terms of giving feedback as well.

Jenny Gordon:

Yeah, and it’s interesting because since the pandemic, and I hate to bring it up, but we see this sort of acceptance of change because we were forced to do so. We couldn’t go to the doctor at some point and we were doing things, we started it with telephone appointments for example, and then perhaps Zoom appointments and things like that. And look at how far we’ve come in just such a short space of time. There’s been an acceptance of change in that sense because it had to happen. But as we talk about the education revolution that’s coming and the change that’s needed and the fear that’s involved in that both at a institutional level and a practitioner level, that could be the biggest challenge to the acceptance and incorporation of authentic assessment without the use of AI, depends the assessment process I guess, and the change needed.

Bert Verhoeven:

Yes.

Jenny Gordon:

What do you see as the sort of biggest challenges when it comes to change? This sort of level of change at institutional and practitioner level?

Bert Verhoeven :

Well, we wrote an article about Bloom who already concluded in the early eighties that a personal attention to students and personal learning leads to much better outcomes than large group learning. And we don’t do anything with it. We at education have been ignoring all these researchers that was out there about experiential learning, leading to higher, better student outcomes, about the personalized learning, leading to better outcomes. All these research has been done that we have ignored because we were able to ignore because of the business model that’s out there now that leads to so much money.

And students are so worried about not having a university degree that the two together adding up has been a perfect situation for universities. I don’t say it is anymore because I think it’s going to change very quickly now and the catalyst will be AI, but this is a much longer process and a much… Bloom was early eighties. This is 40 years, 50 years process that has been happening that university were able to ignore. And I don’t think with AI they can no longer ignore it anymore. It will bring the change that is so needed in higher education.

Vishal Rana:

But I think the challenge also comes down to bureaucracy of the larger universities to implement this change because there’s so many layers and layers of people in the universities and I think the change will drastically happen from the user side of things where the students then say that the industry doesn’t require us to… Because what… I don’t know I should put that in the recording being in higher education. But initially why did we, especially in the countries where we have larger populations, why do you have to go to a university to get a degree? Because it’s this filtering process. Those who have the university degrees get a chance to go for the interviews where that filters out the other people who don’t have the university degrees.

Now the organizations are changing their mindsets and saying, well, you don’t have to go for a three years university degree, you do some short courses, six months. The ones come from Udacity or Coursera, do a six months course and I train in the specific area that we require you to get trained and we’ll hire you. So this is a big question that wasn’t taken on before generative AI, but now with generative AI, it’s going to escalate. And that’s what we call Moore’s law where it’s exponential curve where the rate of change is exponential right now. And actually we are seeing some spaces that it’s double S, double curve, which is in Moore’s law, what it means? The change happens every two years, the things, but now it’s happening double the speed of it. And people in the Silicon Valley are themselves very amazed at the rate of which the technology is learning itself when it comes to learning. So who we are as humans are language. Everything we do is talk through language and if AI can lang anything with language, then everything that we do via it becomes questionable.

And so for the universities to manage that challenge is going to be hard because it’s a disruption that universities will be taken aback as well in the industry. What we are doing right now, every university has a program of internship. They’re all trying to bring, embed that part of which you’re talking about, Jenny, the vocational places where you go and you train and all the universities how they’re trying to [inaudible 00:33:53] internships into their programs in pretty much every program that there is. So then if the students have to actually go and do an internship anyways, why do they have to come to university? The personal tutors can train them in what they want to learn and then they can go and engage with the industry. They can learn everything that the industry needs from them. And so the user then drives the industry and the user doesn’t want to go to university anymore. Then the universities will have to come up with a new lateral thinking as we call it in design thinking, as a totally different way of looking at education.

Jenny Gordon:

Absolutely. That’s sort of user driven mentality. Future students knowing what they want to get out of something. We used the example of my eldest son earlier when we were talking, he was very clear about what he wanted to do, how he was going to get there. And in his case it was a university degree that would do that. He’s just about to start his internship by the way for the summer. But there are others that are looking at their future and they’re thinking, is it a degree? Is it 50, 60, 70,000 pounds of debt, dollars of debt? Am I going to get the value? And that question now that users are asking about the value that they can get from different educational experiences is very important.

And I imagine it’s at the heart of a lot of universities as they consider their future business models and the way in which they intend to continue to grow. What do you think… I mean, the higher education business model as it is today, there’s going to be a shift and I think we’re already starting to see it not just because of AI, but again quality and value, I think at the heart of that conversation. What impact might we see in higher education business models as a result of all of these changes that are happening very quickly?

Bert Verhoeven:

Yeah, I think you’re right to say, Jenny, about user-driven and this is what education will become. So education will become user driven. You want education for the right purpose, for the right what you want to achieve. And that doesn’t mean that you’re going to study for three years about something that you won’t use anymore. So it will be much more user driven and it will be technology driven. So in that sense it’s very comparable to, for example, the music industry that changed dramatically from selling records and making a lot of money from that to not making any money from records being recorded, because it was all download first for free and then it became obviously through all the streaming services, you pay for that now. But still, the amount of money that is in the music industry was for many, many years, a lot less as a total than in the seventies or eighties when these records, when we were young, we were buying them for 20, 30 pounds or 10 pounds, 15 pounds and now you download every song for free.

So there’s a lot less money initially in the industry. And what you see now happening though is most recently, with the organization of these streaming services, the new streaming services, that the revenues go up again and that they’re now almost back again to the levels that they were in the seventies and eighties. And so I think you will see something similar in education where initially there will be a shock and this technology and user driven changes will really lead to something we can’t predict as such. There could be institutions that will really flourish and grow very, very fast.

But I believe initially the total amount of money in the industry as a total will be a lot less because of that and because people will not just go and spend $100,000 for a degree that they will never use anymore. So that is not going to happen anymore with AI as a personal learning tool. And so that will be the initial stage and universities and education will have to adapt to that like the music industry had to, like the newspaper industry had to when they went from print to digital. They will have to adapt to that new situation and new innovations will come up and we can’t predict exactly what will happen, but that disruption is inevitable at the moment. That’s very clear and AI will be the catalyst of that.

Jenny Gordon:

Yeah.

Vishal Rana:

I think actually there could be a lot of universities merging into each other as well to make one big university. We are already seeing that happening in South Australia where two universities are now merging to make one university and I think the suit will follow because I was just reading a report in Queensland in Australia, universities together made a loss of two billion dollars last year, all the universities combined, and two billion dollars is not a small figure. So maybe the future would be starting as a step by step process, could be merging together to make one big university where all the students can come and get all the material they want and then eventually phasing them out as well to a very personalized way of learning. And I think listing from all the things that is going around in this space, I give five to seven years for the universities to exist and probably they will cease to exist because they wouldn’t need a university to go to in the future. That’s what I think.

Jenny Gordon:

And I guess that’s a big statement to make, but I think the combination of, and I say this in floating commas, institutionalized assessment where we have to assess to state that our students have hit a certain level for them then to go out into the workforce, that whole process is changing. So the relationship between education and the workplace is changing as well. And those things together I think are going to come… The development of that passing from education into the workplace is going to become tighter. And that is a place where I think the role of technology and AI together will be fundamental in the changes there. How do we make sure that we are giving our students, our 18 year old students who are just starting in higher education, our 20 year old students, how do we make sure that we are educating them on technology generally as well as AI as best we can for something that we don’t actually know what it looks like yet for when they qualify in two or three years time?

Bert Verhoeven:

I think the only solution is if that we start educating students how to learn. So that will become the core of every course. It doesn’t matter what course you’re doing out there, you need to understand how you learn and how you learn with AI as a copilot. And that will be the most important thing we can teach our 18 year olds at this moment, because then we can use assessment not of learning, but assessments for learning. Because then learning will become the core process and not some knowledge transfer about archeology or about business or marketing. It’s not that anymore. It doesn’t matter, because all the knowledge, AI knows all that. The knowledge is less important in that whole process. It will all become important to learn how to learn, to have the right mindset, the growth mindset for example, to learn about empathy.

So you’re going to be user driven, you understand what the user wants for example, and you learn about AI tools of course, and how to use them. And you learn about how to apply that in reality. So how do you then apply what you have learned into the real world? And that’s where authentic assessment comes in, where authentic assessment really at bare bone is real world assessment and applying it into the world and making mistakes doing that. And you learn again by learning, you learn while doing, and it is assessment for learning instead of testing you how well you know what to do. Everybody would know now about the bar exam in America, how ChatGPT passed that bar exam and was even in the 92 percentile or something. So better than nine out of 10 lawyers to pass that exam.

So this is the kind of knowledge that we cannot compete against. That is the kind of knowledge that Vishal was talking about as well, that doctors can use that we as humans cannot compete against, but we can do though is learn how to learn better and use it to our advantage to make progress. And that’s what we have been doing in entrepreneurship and innovation obviously for a long time. Yeah, it’s always been experiential, it’s always been lean startup-like and then build, measure, learn. We build something, we measure how it goes, prototyping, and then we learn and we do it better again. And in that process, AI is just awesome as a copilot, that is so helpful.

And I think you’ll see that kind of learning in every field and I think that will contribute to much better results in every field again as well. And that in that sense, learning will become agnostic. It will just be very core on what I just described, that those elements are user driven, authentic, and growth mindset. So the [inaudible 00:43:25] motivation for learning, that will be the core of everything we’ll do. And I think then the outcome will be much better at the end of the day, because a doctor will do better than… If a doctor can do that and use ChatGPT as a copilot, well, we’ll all be much happier.

Vishal Rana:

I agree. The growth mindset and the resilient mindset. Because there’s reports coming that the professions, somebody who’s 18 now will probably go to three or four different professions in their lifetime and probably 15 different jobs during their lifetime. So I agree with Bert, it’s about learning for learning, it’s not of learning. So you’re testing knowledge for learning. Because if you are adaptable, you are always keen to learn new things. You’re curious, that growth mindset again comes to being the adaptability of it, the resilience of it. If we can train on the soft skills now, then the professions that don’t exist that will come in the future, students are ready for it. The 18 year olds are ready for it because they know that this is constant change.

I myself changed my profession four times in 20 years and Bert himself has changed his profession a couple of times as well. And I think that’s what makes us more agile and adaptable because we’ve gone through the process ourselves of different professions. And I think that’s the future where if you have that agile mindset, that adaptable mindset, then it becomes easier to go through all these changes that’s happening and evolving and learning new things to be always ready for what’s coming.

Jenny Gordon:

Absolutely, thank you.

Bert Verhoeven:

So last thing is then, so pure knowledge transfer through lectured tutorial is dead. That’s over. We cannot do that anymore. That is pointless. And in the place of that you’ll need experiential learning that goes through the process of the growth mindset with empathy, understanding of the user, and then user-driven kind of improvements and changes in any field. It doesn’t matter where you are, what you’re talking about. And that kind of education is revolutionary, is a real disruption of what is happening now. And because, Vishal said already, it costs more time to develop these programs. It’s more time to teach experiential learning. It’s more personalized learning, the business models will have to change. Yeah, there’s no doubt. So that is I think what’s going to happen. It’s not going to happen in five to 10 years from now, it’s going to happen much faster.

Jenny Gordon:

It’s happening now. And I feel if we spoke in another three or four weeks, there would be more to talk about-

Bert Verhoeven:

There would be.

Jenny Gordon:

On that point. Perhaps we should continue this conversation in regularly, because it’s fascinating to see this much change. Having worked in education technology, I haven’t changed profession, but I have seen these different peaks of introduction of things and personalized learning and [inaudible 00:46:25] and that thing sort of thing, going back to as you said. But many years, so for us to be able to finally take advantage of AI as the catalyst for that, it’ll be great and fascinating and we hope to see significant impact as a result. As we wrap up this brilliant discussion that I thoroughly enjoyed, thank you both, are there any other final concluding points that you would like to mention or if you’d also like to mention your third article that’s being published in a week or so as well? Vishal.

Vishal Rana:

Bret, do you want to go ahead?

Bert Verhoeven:

Yeah, Vishal.

Vishal Rana:

Sure. So I’ll let Bert talk about the third article that’s more dealing with the pedagogical practices that we are writing about in, again, London School of Economics Business Impact Blog. But to wrap up, the change is continuous and I’ll finish it with that analogy about the ostrich. The ostrich sees danger and they just stick their head into the sand hoping that the danger won’t touch them. And I think universities or higher education or any organization for that matter who is going through that ostrich behavior cannot last for too long. And it’s not only about higher education, it’s any organization with knowledge workforce that is trying to close their eyes, seeing that this is inevitable will be disrupted and will be gone much more faster than they could imagine.

Jenny Gordon:

Thank you.

Vishal Rana:

Thank you.

Bert Verhoeven:

Yes, and I totally agree and what I’d like to add about that is that there will be a revolution in education because of AI and I hope that the revolution will go in the right direction. So I hope that universities have the courage to pick it up and to say, okay, we’re not going to resist this anymore. We’re going to drive it and lead to much better outcomes than before instead of just putting our heels in the sand and try to slow things down and stop things. And my hope is that that will happen, that people will say, okay, maybe you know about the Henry Ford once said that, “If I had asked my customers, they would’ve said they wanted to faster horses.”

And I guess that’s what we look at in education. Yeah, if you ask us then we just want better education. We want faster and better and more interesting and less boring and all that kind of education. But I think this is the moment to really rethink education and start thinking about fast car as a solution instead of a faster kind of university or a similar kind of business model that we are doing now. And I really hope that some universities out there will have the courage to say, okay, we’re going to be the leaders in that and we are going to explore that and we’re going to explore these new models that are innovative and different and that may lead to a whole different way of how we educate our children.

Jenny Gordon:

That concludes the brilliant discussion that I had with Bert and Vishal just a couple of days ago, and I hope you’ve enjoyed it as much as I did. There’s been some amazing questions and collaboration taking part within the Q and A panel, and I invite you if you have any further questions to put them there. A lot of colleagues have asked, what will we be sharing after this session? This recording will be shared. We will also link to some of the articles that Bert and Vishal mentioned as part of a series that they have had published within the London School of Economics blog. I wanted also to mention that we at GoReact have recently launched a toolkit that would support your thinking around authentic assessment and how you might consider the use of video within your assessment strategy and techniques. We’ll put a link to that right now in the chat too.

If anybody else has any other questions, please raise them and upvote them if you’d like to. We have had a lot of questions around the validity of information. How do we know that what we’re getting from generative AI tools is correct? How do we know that it’s real, that it’s not fake news? And I think that conversation is one that will go on for a very long time as the use of AI becomes more common within education, but within the world’s everyday use generally. As GoReact, we’ll continue to share information and best practice in these areas to support you as educators with your knowledge and your understanding of AI. If you do have any specific areas of concern or things that you would like us to look at in terms of webinar content that we could host for you or people that we could bring to share their best practice, do let us know and we would be really happy to try and support that as much as we can too.

If there are no other questions or topics of conversation, we’ll get ready to close out for today. We’re always available though to answer your questions. My personal email address is jenny@goreact.com and I welcome those at any time you might have them, but for now from me and all of the GoReact team, thank you so much for joining us today.